over 2000 friends the sql requests are just to much for any server to handle
my server has
6 cpu
8gb ram
and sql cashe set to max memory usage  
at 3000 friends my site becomes useless on try to load chat  

i would hate to see what 500 friends on a shared server would do 

i can see 2 ways of fixing it 

1 oxwall has a cashe system (users friends needs to be cashed)

2 make chat not refresh as users go from page to page like many other site do including facebook