Before you decry the idea of Smoltz on the Yankees because of his horrid 6.35 ERA (with the Red Sox and Cardinals), you should know how unlucky he was. He had a phenomenal K/BB rate of 4/1 (better than any Yankee starter), and a very reasonable FIP of 3.87 (better than any Yankee starter except CC). But an extremely unlucky BABIP of .364 (compared to .293 for his career) upped his ERA to that atrocious number.
Now I wouldn't want him to be a starter. We have four starters who are all above average (both in terms of health and production) and a fifth who should almost certainly surpass Smoltz' production (either Joba or Hughes). Where Smoltz would fit well would be the bullpen. He's closed before with Atlanta, and his fine peripheral numbers from last year would probably increase with a move to the pen (as would his health).
On the flip-side, Smoltz had his highest homerun (1.27/9) and flyball (42%) rates in his career. That's not a good trait for Yankee Stadium. However, his homerun rate was only marginally higher than Joba Chamberlain's, and his flyball rate was actually lower than Phil Hughes'.
The biggest obstacle would be getting him to be a reliever when he seems to want to remain a starter. But if the Yankees can overcome that, and sign him to a reasonable deal (under $4 million), it makes a lot of sense.