I go to Petersfield for books as they have quite a few charity shops which i've found some gems in.
Also there is a great second hand bookshop in Alton, which i would recommend.
Bearing in mind how many times i've been to Albert Road i can not believe i haven't seen this shop, i will be dragging the OH to Southsea soon!