In the course of the day a Frenchman, who was a fugitive from his own side, surrendered to Walley's men, and from him the New England commander learned the somewhat discouraging news that the defensive forces in the city far outnumbered the whole of Phipps's expedition. Troops had been pouring in from different quarters both before and after the governor's arrival, and the last body of men brought by Callières had raised the total to about three thousand. Walley threatened the man very seriously as to what would happen if he did not tell the truth, and he seems to have heeded the warning. The number he mentioned agrees with the figures given by the contemporary historian Belmont, and also by Captain Sylvanus Davis, who was a prisoner in Quebec during the siege.
According to the arrangement made between Phipps and Walley, the former was only to begin the bombardment after the latter had forced an entrance into the town. Moreover, small armed vessels were to sail into the St. Charles, to assist his passage of that river and to furnish his force with necessary supplies of food and ammunition. Why this arrangement was departed from is not very clear; but about four o'clock on Wednesday afternoon Phipps moved his four principal vessels up before the town, and no sooner had he come within cannon shot than the shore batteries opened fire. Then ensued a duel in which the defence had all the best of it. Their guns were much better served than those of the assailants, and they had excellent marks to shoot at. The fight was maintained till after dark, by which time Phipps had fired away nearly all his ammunition and accomplished virtually nothing. One boy in the town had been killed by a splinter of rock; the buildings in the town had scarcely been injured at all. Phipps says he dismounted some of the enemy's best guns, but his story is unconfirmed. Certain it is that his vessels suffered serious damage in hulls, masts, and rigging, and that, after a brief renewal of the encounter the next morning, he drew them all off.