What should have been a relaxed passage through the group stage became a major fear, as a late batting fall down nearly cost Australia exceedingly.
Australia were in control at 4-174 look for 206, but lost 4-13 for six overs, which left them in danger of losing the game and allowing India to overtake them on run-rate.
India is currently playing the West Indies at the Wanderers.
But Australian tail Enders Brett Lee (12 not out) and Nathan Hauritz (9 not out) got their side home when they jumbled through for a bye off the final delivery of the match.
The NSW pair came together needing 19 runs from 25 balls, which became more than a run a ball after Naved-ul-Hasan bowled a maiden in the 47th over.
But Lee and Hauritz shaped the target down, and needing four runs from Umar Gul's last over, got the job done - just.
Scores were level on the final delivery and Hauritz swung and missed, but Lee raced through for the winning single as Pakistan wicketkeeper Kamran Akmal missed with his underarm at the stumps.
The win put Australia top of their group and booked a semi-final against England on Friday, while Pakistan will play New Zealand in the other semi, at the Wanderers on Saturday.
Australia laid the stage for their win by controlling Pakistan to 6-205 on a pitch notable for some variable bounce.
All the Pakistan batsmen got starts, but Watson (2-32), Mitchell Johnson (2-44), Lee (1-30) and Hauritz (0-24) bowled neatly to keep the innings in check after their opponents were 1-58 after 12 overs.
Watson (24) and Paine (29) put Australia on the right track, before Mike Hussey (64) and captain Ricky Ponting (32) had Australia contentedly placed.
But Naved (1-39), Asif Mohammad (2-34) and Saeed Ajmal (2-31) turned the match on its head with a late fight back, until Lee and Hauritz held their nerve.
A win over England will give Australia the chance to become the first side to win back-to-back Champions Trophy tournaments, in Monday's final at Super Sport Park.
Australia will enter Friday's game in a confident frame after beating England 6-1 in the recent one-day series which concluded the Ashes tour.