The author of Hebrews tells us that the Holy Ghost is a witness to us regarding the New Covenant, established by the blood of Jesus.

In chapter 8, he tells us that Jesus is the mediator of a better testament, established on better promises (v.6). The better testament, or covenant, is the New Covenant God promised to the house of Israel and the house of Judah.

The reason the New Covenant was promised is clearly stated: For if that first covenant had been faultless, then should no place have been sought for the second. For finding fault with them, he saith, Behold, the days come, saith the Lord, when I will make a new covenant with the house of Israel and with the house of Judah… ~ Hebrews 8:7-8.

The fault that God found with them was that they “continued not” in his covenant, and this was the result of sin. Israel’s covenant with God had no power to end the dominance of sin over his people, therefore, God promised to make a new covenant.

For this is the covenant that I will make with the house of Israel after those days, saith the Lord; I will put my laws into their mind, and write them in their hearts: and I will be to them a God, and they shall be to me a people: And they shall not teach every man his neighbour, and every man his brother, saying, Know the Lord: for all shall know me, from the least to the greatest. For I will be merciful to their unrighteousness, and their sins and their iniquities will I remember no more. ~Hebrews 8:10-12

Though Israel was given the Law, the tabernacle, the priesthood, and the atonement sacrifices, God was not pleased because there was no true cleansing from sins,

For the law having a shadow of good things to come, and not the very image of the things, can never with those sacrifices which they offered year by year continually make the comers thereunto perfect. For then would they not have ceased to be offered? because that the worshippers once purged should have had no more conscience of sins. But in those sacrifices there is a remembrance again made of sins every year. For it is not possible that the blood of bulls and of goats should take away sins. ~ Hebrews 10:1-4

The sacrifices on which the Old Covenant was established, and which were offered for sins, were a continual reminder that sin had not been permanently dealt with, and thus, there was no true reconciliation between God and his people.

Then verily the first covenant had also ordinances of divine service, and a worldly sanctuary. For there was a tabernacle made; the first, wherein was the candlestick, and the table, and the shewbread; which is called the sanctuary. And after the second veil, the tabernacle which is called the Holiest of all; Which had the golden censer, and the ark of the covenant overlaid round about with gold, wherein was the golden pot that had manna, and Aaron’s rod that budded, and the tables of the covenant; And over it the cherubims of glory shadowing the mercy seat; of which we cannot now speak particularly. Now when these things were thus ordained, the priests went always into the first tabernacle, accomplishing the service of God. But into the second went the high priest alone once every year, not without blood, which he offered for himself, and for the errors of the people: The Holy Ghost this signifying, that the way into the holiest of all was not yet made manifest, while as the first tabernacle was yet standing… ~ Hebrews 9:1-8

In contrast to the ineffectiveness of the first covenant, the author of Hebrews expounds on the new. He tells us, the Holy Spirit is a witness to us of the New Covenant, established in the blood of Jesus.

Then said he, Lo, I come to do thy will, O God. He taketh away the first, that he may establish the second. By the which will we are sanctified through the offering of the body of Jesus Christ once for all. And every priest standeth daily ministering and offering oftentimes the same sacrifices, which can never take away sins: But this man, after he had offered one sacrifice for sins for ever, sat down on the right hand of God; From henceforth expecting till his enemies be made his footstool. For by one offering he hath perfected for ever them that are sanctified. Whereof the Holy Ghost also is a witness to us: for after that he had said before, This is the covenant that I will make with them after those days, saith the Lord, I will put my laws into their hearts, and in their minds will I write them; And their sins and iniquities will I remember no more. Now where remission of these is, there is no more offering for sin. ~ Hebrews 10:9-18

Contextually, the putting away of sins and the once for all references in the book of Hebrews, have to do with the permanency of the finished work of Christ in contrast to the imperfect atonement sacrifices under the Law. Because of Jesus, the New Covenant is superior to the Old.

Though the first covenant was ineffective in taking away sins, God gave it to Israel to prepare the people for something much greater. It would serve as a foreshadowing of Christ, who was yet to come.   

But Christ being come an high priest of good things to come, by a greater and more perfect tabernacle, not made with hands, that is to say, not of this building; Neither by the blood of goats and calves, but by his own blood he entered in once into the holy place, having obtained eternal redemption for us. ~ Hebrews 9:11-12

At best, the old covenant could only serve as a foreshadowing of the new. The first covenant, established with the blood of animal sacrifices, was insufficient for the cleansing of sins. Therefore, it had no power to reconcile man to God.

Through his blood, Jesus has reconciled us to God. As our Great High Priest, he has given us unhindered access into the presence of the Living God, because his blood cleanses us from all sin.

Having therefore, brethren, boldness to enter into the holiest by the blood of Jesus, By a new and living way, which he hath consecrated for us, through the veil, that is to say, his flesh; And having an high priest over the house of God; Let us draw near with a true heart in full assurance of faith, having our hearts sprinkled from an evil conscience, and our bodies washed with pure water. ~ Hebrews 10:19-22

This is the covenant, to which the Holy Spirit bears witness to our hearts!