A question about linking

I added a link to my site which was ranked in the first page at google, now it is in the third page. Will the site return to the first page if I remove the link?