Youngster Barrie McKay opened his Rangers account with what should have been a goal to savour on Saturday, before second-half goals from Rory McAllister and McLaughlin saw the home side surge into the lead.
Andy Little then grabbed a late equaliser and a share of the points for Ally McCoist's men.
"If you get ahead in the game, you think you're going to win it," McLaughlin told Sky Sports One.
"We're a little disappointed with the result at the end, although before the game we would have taken a draw any day.
"We always knew they were going to come at us a bit but we gave a great account of ourselves and we can be proud."
McAllister also took some positives from the game, saying: "We've got a good chance of chasing them for the rest of the season if we can keep playing the way we are."
Rangers manager Ally McCoist was unhappy with his side's defending, which he felt cost them three points in their opening match in the Irn-Bru Third Division.
"I was relieved to get a point after the game," he said.
"I thought we defended appallingly to be honest. We didn't match their aggression.
"If the boys at any time thought this was going to be a cake-walk, they know where they are now."
He added: "It's the first game, we're not going to start panicking. Poor defending is a concern but we can certainly create and score goals.
"I have to give enormous credit to Peterhead and (manager) Jim McInally. They played superbly today.
Rangers defender Carlos Bocanegra was also impressed by Peterhead's display.
"It was a tough game," said the American.
"They fought and kicked until the end and that's something we can expect all season, but we have to give credit to them.
"We had a few chances but we didn't finish them. It's frustrating but that's probably what it's going to be like this season.
"We need to play better. We got ourselves in trouble a little bit with our passing, we didn't finish opportunities and we didn't defend well either, so we've got to clean those things up."
The new PUMA evoSPEED 1.2 FG. A great new soccer cleat with classic look and first time for PUMA to be on the cleat.
Borussia Dortmund hears Wembley Calling and are headed to London for the Champions League final.
One of the top soccer cleats for control the Nike CTR360 will not let you down on the field in tight spaces.
adidas has added some color to the Predator line with this new colorway. Released on May 17. Be the first to own a pair.
Chelsea won the Europa League on Wednesday. The Blues defeated Benfica 2-1 on injury time goal from Ivanovic.
The Juventus jersey for the 2013/14 was released earlier this week. Be the first to wear the colors of the Old Lady.
The Liverpool 2013/14 soccer jersey is available for pre-order. The Reds are eager to compete for silver next season.
Ajax has released their away soccer jersey for the 2013/14 season. Get in the Gear.
Bayern Munich are kings of Germany and hope to be crowned kings of Europe at Wembley later this month!
The push for more color on their cleats, adidas has added the infrared colorway to the adiZero f50 Line.