Tunisia goalkeeper Mouez Hassen made back-to-back saves inside a minute.
However, at the time of writing, England could do with another close range finish from Kane to ensure they get off to the desired start in Russian Federation.
With a cross coming into the box, the Manchester City defender threw his arm up and caught the Tunisia attacker in the face, with Ben Youssef immediately going to ground.
After Belgium thrashed Panama 3-0 in the other Group G game on Monday, the pressure was on England to deliver only their second win in their last nine games at World Cups.More news: Former FBI Boss Comey Used Personal Email During Clinton Case
More news: U.S. soybean prices tumble amid trade fight with Beijing
More news: England opt for attack-focused midfield against Tunisia
Tunisia started the second-half brightly and seemed set for a memorable draw, but ran out of steam as England pressed in the closing stages.
But what Southgate's men lack in experience they make up for in spirit, with Tottenham striker Kane nodding home at the far post to clinch a dramatic late victory which sent their hardy travelling support wild.
"We deserved the win".
It was relief for Kane and Gareth Southgate, who leapt into the air in delight as his World Cup debut as a coach got off to a winning start.
The squad, dominated by local based players, doesn't include influential midfielder Youssef Msakni and the Tunisian league's top scorer Taha Yassine Khenissi, who have both been forced out by injuries. Tunisia looked more comfortable as they sat back in defense, only to be undone by Kane in the 91st minute. England allowed carelessness to creep in at the back and offered a limited Tunisia team the chance to get back into a game they should have been out of within the opening half-hour.