Research

This project started as a research project to find out how to create a common interface for different search engines. In this document we collect all the information we found out during our research. Feel free to add all kind of interesting information you want to share.

List of Search Engines

Here we collect different search engines which are around and could be interesting:

• Elasticsearch - cmsig/seal-elasticsearch-adapter

• Opensearch - cmsig/seal-opensearch-adapter

• Meilisearch - cmsig/seal-meilisearch-adapter

• Algolia - cmsig/seal-algolia-adapter

• Solr - cmsig/seal-solr-adapter

• RediSearch - cmsig/seal-redisearch-adapter

• Typesense - cmsig/seal-typesense-adapter

• Loupe - cmsig/seal-loupe-adapter

• Zinc Labs (work in progress #79)

• Manticore Search (work in progress #103)

• ZendSearch

• Kailua Labs

• TnTSearch

• Sonic

• Vespa

• Toshi

• Quickwit

• nrtSearch

• MongoDB Atlas

• PostgreSQL Full Text Search

• MySQL Full Text Search

• Sphinx Search

• Search.io

• Azure Cognitive Search

• Google Cloud Search

• Amazon CloudSearch

• Gigablast

• Fess

• Bleve

• Qdrant

• OpenAI

• Jina

• Paradedb

Some more research links:

• https://alternativeto.net/software/meilisearch/

• https://github.com/awesome-selfhosted/awesome-selfhosted#search-engines

• https://help.openai.com/en/articles/6272952-search-transition-guide

• https://www.reddit.com/r/PHP/comments/104278m/research_what_search_services_engines_do_you_use/

• https://github.com/doofinder/php-doofinder

• https://www.athenasearch.io/

• https://www.g2.com/products/addsearch-site-search/reviews

• https://aws.amazon.com/de/athena/ / https://twitter.com/dr4goonis/status/1628451049013972993

• https://milvus.io/ / https://twitter.com/milvusio / https://packagist.org/packages/kaycn/milvusphp

• https://github.com/pgvector/pgvector

• https://vald.vdaas.org/

• https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html

• https://github.com/facebookresearch/faiss

• https://blog.vespa.ai/billion-scale-knn/

UI/UX related links:

• https://design4users.com/design-search-in-user-interfaces/

Optimization links:

• https://sites.google.com/site/kevinbouge/stopwords-lists

• https://github.com/uschindler/german-decompounder

• https://symfony.com/blog/migrating-symfony-com-search-engine-to-meilisearch

• https://thetizzo.com/2024/02/13/scaling-elasticsearch-for-fun-and-profit

Descriptions of Search Engines

Elasticsearch

Widely used search based on Java.

• Server: Elasticsearch Server

• PHP Client: Elasticsearch PHP

Implementation: cmsig/seal-elasticsearch-adapter

Opensearch

Fork of Elasticsearch also written in Java.

• Server: Opensearch Server

• PHP Client: Opensearch PHP

Implementation: cmsig/seal-opensearch-adapter

Meilisearch

A search engine written in Rust:

• Server: MeiliSearch Server

• PHP Client: MeiliSearch PHP

Implementation: cmsig/seal-meilisearch-adapter

Algolia

Is a search as SaaS provided via Rest APIs and SDKs:

• Server: No server only Saas https://www.algolia.com/

• PHP Client: Algolia PHP

Implementation: cmsig/seal-algolia-adapter

Solr

A search engine under the Apache Project based on Lucene written in Java:

• Server: Solr Server

• PHP Client: Solarium PHP seems to be a well maintained Client

Implementation: cmsig/seal-solr-adapter

RediSearch

A search out of the house of the redis labs.

• Server: RediSearch Server

• PHP Client: Unofficial RediSearch PHP

Implementation: cmsig/seal-redisearch-adapter

Typesense

Describes itself as a alternative to Algolia and Elasticsearch written in C++.

• Server: Typesense Server

• PHP Client: Typesense PHP

Implementation: cmsig/seal-typesense-adapter

Loupe

An SQLite based, PHP-only fulltext search engine.

• Implementation: Loupe PHP

Zinc Labs

Zinc search describes itself as a lightweight alternative to Elasticsearch written in GoLang.

• Server: Zinclabs Server

• PHP Client: No PHP SDK currently: https://github.com/zinclabs/zinc/issues/12

Implementation: work in progress #79

Manticore Search

Fork of Sphinx 2.3.2 in 2017, describes itself as an easy to use open source fast database for search. Good alternative for Elasticsearch.

• Server: Manticore Search Server

• PHP Client: Manticore Search PHP Client

Implementation: work in progress #103

ZendSearch

A complete in PHP written implementation of the Lucene index. Not longer maintained:

• Implementation: Zendsearch Implementation

Kailua Labs

Next-gen search made simple:

• Server: No server only SaaS https://www.kailualabs.com/

TnTSearch

Another implementation of a Search index written in PHP. Not based on Lucene.

• Implementation: TntSearch Implementation

Sonic

Describe itself as lightweight & schema-less search backend, an alternative to Elasticsearch that runs on a few MBs of RAM.

• Server: Sonic Server

• PHP Client: Unoffical PHP Sonic looks outdated and not well maintained

Vespa

Describe itself as the open big data serving engine - Store, search, organize and make machine-learned inferences over big data at serving time.

• Server: Vespa Server

• PHP Client: No client available only API based

Toshi

A full-text search engine in rust. Toshi strives to be to Elasticsearch what Tantivy Server is to Lucene:

• Server: Toshi Server

• PHP Client: No client available only API based

Quickwit

Describe itself as a cloud-native search engine for log management & analytics written in Rust. It is designed to be very cost-effective, easy to operate, and scale to petabytes.

• Server: Quickwit Server

• PHP Client: No client available only API based

nrtSearch

Describe itself as a high performance gRPC server, with optional REST APIs on top of Apache Lucene version 8.x source, exposing Lucene’s core functionality over a simple gRPC based API.

• Server: nrtSearch Server

• PHP Client: No client available only API based

MongoDB Atlas

None open source search engine from MongoDB. It is a cloud based search engine.

• Server: MongoDB Atlas

• PHP Client: MongoDB Atlas PHP Client

PostgreSQL Full Text Search

• Server: PostgreSQL Server

• PHP Client: No client use the Full Text Feature the Database connection.

MySQL Full Text Search

• Server: MySQL Server

• PHP Client: No client use the Full Text Feature the Database connection.

Sphinx Search

An older search engine written in Python.

• Server: Sphinx Search Server

• PHP Client: No official client available

Search.io

A SaaS search engine, In the past they used the name for Sajari Site Search. Lately acquired by Algolia.

• Server: No server only Saas Search.io Server

• PHP Client: Official Search.io SDK for PHP

Azure Cognitive Search

A cloud based search from Microsoft Azure:

• Server: No server only SaaS Azure Cognitive Search

• PHP Client: No client available only REST API

Google Cloud Search

A cloud based search from Google:

• Server: No server only SaaS Google Cloud Search

• PHP Client: No client available only REST API

Amazon CloudSearch

A cloud based search from Amazon:

• Server: No server only SaaS Amazon CloudSearch

• PHP Client: No client available only REST API

Gigablast

Describe itself as an open source web and enterprise search engine and spider/crawler written in C++.

• Server: Gigablast Server

• PHP Client: No client available only REST API

Fess

Fess is very powerful and easily deployable Enterprise Search Server.

• Server: Fess Server

Bleve

A modern text ndexing in go, supported and sponsored by Couchbase:

• Library only: Bleve

Qdrant

A vector AI based search database:

• Server: Qdrant Server

• PHP Client: No client available only REST API

OpenAI

OpenAi embeddings can also be used to create search engine:

• Docs Embeddings: Embeddings

• Docs Search: Deprecated Search Migratin Transition

Jina

Another vector based search engine:

• Server: Jina Server

Paradedb

A search and analytics engine ontop of Postgres, with own Postgres extensions written in Rust:

• Server: Paradedb Server