Two goals in three minutes on the stroke of half-time meant the Seasiders trailed 2-0 at the break before Dan Rowe scrambled home what would prove to be only a consolation midway through the second half.
The defeat means that the Seasiders remain rooted to the foot of the Pitching In Northern Premier League Premier Divison table without a point to their name.
And Bullock was frustrated by more mistakes that eventually led to another two goals in the against column for Whitby.
"We rode our luck in the first ten minutes," Bullock said post-match
.
"We made three terrible mistakes and got away with it thanks to a couple of good saves from Shane Bland.
"For the next thirty odd minutes, however, we were happy. It was a good away performance but then before half-time we may as well have just pressed play from Saturday.
"We were level on 44 minutes, then we're 2-0 down at half-time. It's criminal.
"We should be seeing ourselves through to half-time, but then we've allowed a free header and a loose pass on the edge of our box and all of a sudden it's game over. It's very difficult to come back from that.
"The lads have been told that they've got to learn and grow up quickly because every bit of weakness is being exposed."
The opening stages of the game were littered with said mistakes for the Seasiders. Aaron Cunningham sliced a ball into the path of Donald Chimalilo, before recovering well to head the Guiseley forward's effort off the line.
And minutes later there was another let off for the Blues when Shane Bland saved well from John Johnston's effort, after Jonny Burn's mistake let the lively Lions wide man in.
Andy Kellett was the next with a warning for Whitby, drilling a powerful effort over the crossbar just inside the area, before Shane Bland held a tame effort from Luke Dean.
It was the recently-relegated hosts who were creating the better of the chances in the opening 45 minutes, particularly through bustling forward Jake Cassidy, who put the ball onto the roof of the net, then into Shane Bland's hands, in the space of just a minute.
At the other end, Nicky Walker slipped Marcus Giles through for Whitby, though the forward couldn't quite beat an onrushing Kyle Trennery to the ball.
Just a minute before the break, Whitby were the masters of their own undoing, allowing defender Spencer Harris a free header at the back post from a corner to open the scoring.
And two minutes later it went from bad to worse for the Seasiders. Whitby turned possession over, allowing Guiseley to get the ball into Cassidy, who got away from his man and fired across Bland just inside the box.
Whitby perked up after the break despite the setback, with Jake Hackett firing a free-kick over the bar early into second half proceedings.
However, the Lions remained dangerous, with Cunningham forced to clear off the line for a second time on the night following John Johnston's cut back.
Just a couple of minutes of later though Whitby had a lifeline. Guiseley failed to clear a corner and Dan Rowe reacted quickest in the melee to scramble the ball into the back of the net.
As the game wore on, Whitby continued to push for a leveler, with the home crowd calling for any free-kick to slow the game down and halt Whitby's progress at any opportunity.
Cunningham flicked an effort goalwards from Rowe's knock down inside the box, before Hawkins saw a drilled effort blocked before Trennery got a hold of the ball.
Lions substitute Lewis Hay stung the palms of Shane Bland two minutes later with the hosts looking to kill the game off.
The Whitby gloveman was back in action just a minute later, saving Cassidy's effort well, before Hey drilled an effort wide from a tight angle after the ball was returned back into the box.
In the dying embers of the game, Whitby came desperately close to leveling. Aaron Haswell picked up the ball and drove into the penalty area before drilling mere inches wide of the mark.
Priestley Griffiths also had a go at finding an equaliser too, powering a low effort wide of the right-hand upright from range.
The Seasiders will be looking to pick up their first points of the season in a big bank holiday weekend, when second-placed Stafford Rangers visit the Towbar Express Stadium at the Turnbull Ground, before they travel to table toppers Marske United on Monday afternoon.