The guys they had behind him certainly didn't help, but lets not forget that for the first half of the season, this was a secondary that was ranked in the top 5 statistically, I think at their peak they were #3 statistically speaking. Shareece Wright was having bad games even with Jimmy in the line up, most notably Oakland and Jacksonville, he was benched after and the team still put a secondary that can be argued as a top 5 secondary overall. The fact that the Ravens didn't have good play at all at #2 and were still holding it down against the passing game speaks for Jimmy's worth. Credit to Weddle and Webb for fantastic play in the backfield but they couldn't save this secondary when Jimmy went out.
As far as a complimentary guy goes to step up in Jimmy's absence, that's fine but if you're the Ravens then I think you hope that Jimmy can play a full season for you. He's too good of a player and talent that you have to rely upon him. I posted some statistics up above of what the Ravens had with Jimmy in the line up and it just shows what the guy can do for you. As I said, taking the cap number I have into account and what we lost with him, I can honestly say that cap number that we're taking from him is worth it if that's his value.
I like Tavon as well, he was a guy who played bigger than his size in College but against guys like Dez Bryant and Branden LaFell, he wasn't really beaten by extraordinary route running or burnt in any manner, he was beaten by being physically taken to school. I think Young has to prove that he can match up with bigger receivers in the future. He might have to put on more weight. Chris Harris Jr(And I'm only comparing them in terms of size) is not the biggest CB out there but he's pretty damn strong at 200 LBs and allows opposing receivers to beat him by height.
Harbaugh said this and I agree, the Ravens need two things at CB, Jimmy Smith to be healthy and another CB.