Swiftype is a custom site search engine as a service. They have a crawler that will index your site, but they also provide an API you can use to publish directly to their search index. You may want to do this to make sure your content is always up-to-date, if not all your content is crawlable, or just to have complete control over what gets indexed.
Here's an 8 minute long screencast I made showing how to do that with a Rails application.
The code is available on GitHub.