Okay so I am building this website which has functionality similar to facebook and twitter and it has highly dynamic content.
All the questions etc, and search engines I have seen are that which create index and are therefore better suited with a site with more or less static content.
I need some recommendations and advice on how to use these index based search engines with a highly dynamic website considering new users will be joining every hour, new content generated, and content being edited. It is quite clear that rebuilding an index every time just to keep the search from going stale is rather absurd.
One solution that came close to solving this issue was using MySQL MYISAM FULLTEXT search columns but I really need an alternate to that because the lack of foreign keys leads to data redundancy, plus I need something that can scale as the website grows and be flexible to customized algorithms for ranking etc.
Thanks..