OOMOut Of MemoryLearn. Share. Succeed.
← All articles
Couchbase

Couchbase - Full Text Search (FTS)

By Sagar Chaudhari2 min read

Couchbase Server stores JSON documents in buckets and provides multiple ways to query that data. N1QL is useful for structured queries over document fields, while Full-Text Search (FTS) is designed for search-oriented use cases such as matching terms and phrases, prefix searches, Boolean combinations, and range queries.

Version context: This article was written in August 2017, when the FTS capability described here was still presented as a developer-preview feature. Product terminology, user-interface steps, APIs, and integration with other Couchbase query capabilities have evolved since then. The article is preserved as a practical example of how the feature was configured and used at that time.

What FTS Adds

Compared with a structured N1QL query, FTS supports search patterns such as:

  • Term, phrase, match, match-phrase, and prefix queries
  • Conjunction, disjunction, and Boolean queries
  • Numeric and date ranges
  • Query-string expressions

Using FTS requires a Couchbase node with the search service enabled and at least one search index over the relevant documents.

Create a Search Index

In this example, documents contain a JSON field named type, and separate type mappings are used to control which documents are indexed.

To create an index for documents whose type is user:

  1. Sign in to the Couchbase Web Console.
  2. Open Indexes → Full Text → New Full Text Index.
  3. Name the index fts-user-index.
  4. Select the target bucket.
  5. Configure type as the Type Identifier field.
  6. Under Type Mappings, add a mapping named user.
  7. Keep inheritance enabled for the mapping and configure the indexed fields as required.
  8. Disable the default type mapping if only explicitly configured document types should be indexed.
  9. Create the index.

The important design choice here is the type discriminator. If a bucket stores multiple logical document types, using a stable field such as type allows search indexes to target only the documents relevant to a particular use case.

Query FTS from Java

The original implementation used the Couchbase Java SDK to execute several FTS operations programmatically.



package com.sample.fts;


import java.util.ArrayList;
import java.util.List;


import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.Validate;


import com.couchbase.client.java.Bucket;
import com.couchbase.client.java.search.SearchQuery;
import com.couchbase.client.java.search.queries.AbstractFtsQuery;
import com.couchbase.client.java.search.queries.BooleanQuery;
import com.couchbase.client.java.search.queries.ConjunctionQuery;
import com.couchbase.client.java.search.queries.DisjunctionQuery;
import com.couchbase.client.java.search.result.SearchQueryResult;


/**
 * This class provides various user friendly static methods to query Couchbase
 * with FTS (Full-Text-Search) feature.
 * 
 * @author Sagar Chaudhari
 */
public class CouchbaseFullTextService {
	
	private static final String FIELD_SEPARATOR = ":";
	private static final String TERM_TRUE = "T";
	private static final String TERM_FALSE = "F";
	
	private CouchbaseFullTextService() {
		// Private Constructor
	}


	/*-
	 * <p> Find records with one or more matching texts.</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * CouchbaseFullTextService.findByMatchingTexts(bucket, INDEX_NAME, 0, false, "attributes.attr1:attr", "attributes.attr:sample");
	 * CouchbaseFullTextService.findByMatchingTexts(bucket, INDEX_NAME, 0, true, "attributes.attr1:attr", "attributes.attr:sample");
	 * CouchbaseFullTextService.findByMatchingTexts(bucket, INDEX_NAME, 0, false, "attributes.attr1:attr", "sample");
	 * CouchbaseFullTextService.findByMatchingTexts(bucket, INDEX_NAME, 0, true, "attributes.attr1:attr", "sample");
	 * CouchbaseFullTextService.findByMatchingTexts(bucket, INDEX_NAME, 0, false, "sample");
	 * CouchbaseFullTextService.findByMatchingTexts(bucket, INDEX_NAME, 0, true, "sample");
	 * CouchbaseFullTextService.findByMatchingTexts(bucket, INDEX_NAME, 2, false, "sample");
	 * CouchbaseFullTextService.findByMatchingTexts(bucket, INDEX_NAME, 2, true, "sample");
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param fuzziness  the fuzziness (default 0)
	 * @param matchAll  if true, then return results which has all the matching searchTexts; if false, then return results which has at any matching searchTexts; default false 
	 * @param searchTexts  one or more search texts
	 * @return  SearchQueryResult which contains document ids and hit locations
	 */
	public static SearchQueryResult findByMatchingTexts(Bucket bucket, String searchIndex, int fuzziness, boolean matchAll, String... searchTexts) {
		Validate.notEmpty(searchTexts, "Value of %s cannot be null or empty", "searchTexts");
		
		List<AbstractFtsQuery> queries = new ArrayList<AbstractFtsQuery>();
		for (String searchText : searchTexts) {
			String[] searchTextArr = StringUtils.split(searchText, FIELD_SEPARATOR, 2);
			if (searchTextArr.length == 2) {
				queries.add(SearchQuery.match(searchTextArr[1]).field(searchTextArr[0]).fuzziness(fuzziness));
			} else {
				queries.add(SearchQuery.match(searchText).fuzziness(fuzziness));
			}
		}
		
		AbstractFtsQuery[] abstractFtsQueries = new AbstractFtsQuery[queries.size()];
		abstractFtsQueries = queries.toArray(abstractFtsQueries);
		if (matchAll) {
			return bucket.query(new SearchQuery(searchIndex, new ConjunctionQuery(abstractFtsQueries)));
		} else {
			return bucket.query(new SearchQuery(searchIndex, new DisjunctionQuery(abstractFtsQueries)));
		}
	}
	
	/*-
	 * <p> Find records with one or more matching phrases.</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * CouchbaseFullTextService.findByMatchingPhrases(bucket, INDEX_NAME, false, "attributes.attr1:sample attr", "attributes.attr:another value");
	 * CouchbaseFullTextService.findByMatchingPhrases(bucket, INDEX_NAME, true, "attributes.attr1:sample attr", "attributes.attr:another value");
	 * CouchbaseFullTextService.findByMatchingPhrases(bucket, INDEX_NAME, false, "attributes.attr1:sample attr", "another value");
	 * CouchbaseFullTextService.findByMatchingPhrases(bucket, INDEX_NAME, true, "attributes.attr1:sample attr", "another value");
	 * CouchbaseFullTextService.findByMatchingPhrases(bucket, INDEX_NAME, false, "sample attr");
	 * CouchbaseFullTextService.findByMatchingPhrases(bucket, INDEX_NAME, true, "sample attr");
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param matchAll  if true, then return results which has all the matching searchTexts; if false, then return results which has at any matching searchTexts; default false
	 * @param searchPhrases  one or more search phrases
	 * @return  SearchQueryResult which contains document ids and hit locations
	 */
	public static SearchQueryResult findByMatchingPhrases(Bucket bucket, String searchIndex, boolean matchAll, String... searchPhrases) {
		Validate.notEmpty(searchPhrases, "Value of %s cannot be null or empty", "searchPhrases");
		
		List<AbstractFtsQuery> queries = new ArrayList<AbstractFtsQuery>();
		for (String searchPhrase : searchPhrases) {
			String[] searchPhraseArr = StringUtils.split(searchPhrase, FIELD_SEPARATOR, 2);
			if (searchPhraseArr.length == 2) {
				queries.add(SearchQuery.matchPhrase(searchPhraseArr[1]).field(searchPhraseArr[0]));
			} else {
				queries.add(SearchQuery.matchPhrase(searchPhrase));
			}
		}
		
		AbstractFtsQuery[] abstractFtsQueries = new AbstractFtsQuery[queries.size()];
		abstractFtsQueries = queries.toArray(abstractFtsQueries);
		if (matchAll) {
			return bucket.query(new SearchQuery(searchIndex, new ConjunctionQuery(abstractFtsQueries)));
		} else {
			return bucket.query(new SearchQuery(searchIndex, new DisjunctionQuery(abstractFtsQueries)));
		}
	}
	
	/*-
	 * <p> Find records by regular expression.</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * CouchbaseFullTextService.findByRegularExpression(bucket, INDEX_NAME, "[a-z]*\\s*attr");
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param searchExpression  the regular expression
	 * @return  SearchQueryResult which contains document ids and hit locations
	 */
	public static SearchQueryResult findByRegularExpression(Bucket bucket, String searchIndex, String searchExpression) {
		Validate.notEmpty(searchExpression, "Value of %s cannot be null or empty", "searchExpression");
		return bucket.query(new SearchQuery(searchIndex, SearchQuery.regexp(searchExpression)));
	}
	
	/*-
	 * <p> Find records by prefix i.e. any word starting with.</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * CouchbaseFullTextService.findByPrefix(bucket, INDEX_NAME, "sample");
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param searchPrefix  the search prefix
	 * @return  SearchQueryResult which contains document ids and hit locations
	 */
	public static SearchQueryResult findByPrefix(Bucket bucket, String searchIndex, String searchPrefix) {
		Validate.notEmpty(searchPrefix, "Value of %s cannot be null or empty", "searchPrefix");
		return bucket.query(new SearchQuery(searchIndex, SearchQuery.prefix(searchPrefix)));
	}
	
	/*-
	 * <p> Find records by text with wild cards (* or ?)</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * CouchbaseFullTextService.findByWildcard(bucket, INDEX_NAME, "sample*");
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param searchWildcard  the search text with wild card (* or ?)
	 * @return  SearchQueryResult which contains document ids and hit locations
	 */
	public static SearchQueryResult findByWildcard(Bucket bucket, String searchIndex, String searchWildcard) {
		Validate.notEmpty(searchWildcard, "Value of %s cannot be null or empty", "searchWildcard");
		return bucket.query(new SearchQuery(searchIndex, SearchQuery.wildcard(searchWildcard)));
	}
	
	/*-
	 * <p> Find records by boolean value</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * CouchbaseFullTextService.findByBoolean(bucket, INDEX_NAME, "isActive", false);
	 * CouchbaseFullTextService.findByBoolean(bucket, INDEX_NAME, "isActive", true);
	 * CouchbaseFullTextService.findByBoolean(bucket, INDEX_NAME, null, true);
	 * CouchbaseFullTextService.findByBoolean(bucket, INDEX_NAME, "", true);
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param field  the field name (ignore if null or empty)
	 * @param searchValue  search value true/false
	 * @return  SearchQueryResult which contains document ids and hit locations (locations will be empty in this case)
	 */
	public static SearchQueryResult findByBoolean(Bucket bucket, String searchIndex, String field, boolean searchValue) {
		/*-
		 * NOTE: Looks like there is a bug with .booleanField implementation.
		 * It throws exception: com.couchbase.client.java.error.FtsMalformedRequestException: FTS request is malformed.
		 * So, .booleanField syntax cannot be used. Commenting.
		 * 
		 * Work around is to use .term instead. Internally boolean values are stored as T/F.
		 * See below.
		 */
		
		//return bucket.query(new SearchQuery(searchIndex, SearchQuery.booleanField(searchValue).field(field)));
		
		return bucket.query(new SearchQuery(searchIndex, SearchQuery.term(searchValue ? TERM_TRUE : TERM_FALSE).field(field)));
	}
	
	/*-
	 * <p>Find records by various combinations</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * CouchbaseFullTextService.findByMatchingNonMatchingTexts(bucket, INDEX_NAME, new String[] {"attributes.supporterId:RegressionSupporter1"}, new String[] {"attributes.attr:attr"}, new String[] {"sample"});
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param searchMustTexts  search text(s) which must match
	 * @param searchMustNotTexts  search text(s) which must not match
	 * @param searchShouldTexts  search text(s) which should be available (may or may not be available in the response)
	 * @return  SearchQueryResult which contains document ids and hit locations
	 */
	public static SearchQueryResult findByMatchingNonMatchingTexts(Bucket bucket, String searchIndex, String[] searchMustTexts, String[] searchMustNotTexts, String[] searchShouldTexts) {
		
		BooleanQuery query = SearchQuery.booleans();
		List<AbstractFtsQuery> mustQueries = new ArrayList<AbstractFtsQuery>();
		List<AbstractFtsQuery> mustNotQueries = new ArrayList<AbstractFtsQuery>();
		List<AbstractFtsQuery> shouldQueries = new ArrayList<AbstractFtsQuery>();
		
		if (searchMustTexts != null && searchMustTexts.length > 0) {
			for (String searchMustText : searchMustTexts) {
				String[] searchMustTextArr = StringUtils.split(searchMustText, FIELD_SEPARATOR, 2);
				if (searchMustTextArr.length == 2) {
					mustQueries.add(SearchQuery.match(searchMustTextArr[1]).field(searchMustTextArr[0]));
				} else {
					mustQueries.add(SearchQuery.match(searchMustText));
				}
			}
			
			AbstractFtsQuery[] abstractFtsQueries = new AbstractFtsQuery[mustQueries.size()];
			abstractFtsQueries = mustQueries.toArray(abstractFtsQueries);
			query = query.must(abstractFtsQueries);
		}
		
		if (searchMustNotTexts != null && searchMustNotTexts.length > 0) {
			for (String searchMustNotText : searchMustNotTexts) {
				String[] searchMustNotTextArr = StringUtils.split(searchMustNotText, FIELD_SEPARATOR, 2);
				if (searchMustNotTextArr.length == 2) {
					mustNotQueries.add(SearchQuery.match(searchMustNotTextArr[1]).field(searchMustNotTextArr[0]));
				} else {
					mustNotQueries.add(SearchQuery.match(searchMustNotText));
				}
			}
			
			AbstractFtsQuery[] abstractFtsQueries = new AbstractFtsQuery[mustNotQueries.size()];
			abstractFtsQueries = mustNotQueries.toArray(abstractFtsQueries);
			query = query.mustNot(abstractFtsQueries);
		}
		
		if (searchShouldTexts != null && searchShouldTexts.length > 0) {
			for (String searchShouldText : searchShouldTexts) {
				String[] searchShouldTextArr = StringUtils.split(searchShouldText, FIELD_SEPARATOR, 2);
				if (searchShouldTextArr.length == 2) {
					shouldQueries.add(SearchQuery.match(searchShouldTextArr[1]).field(searchShouldTextArr[0]));
				} else {
					shouldQueries.add(SearchQuery.match(searchShouldText));
				}
			}
			
			AbstractFtsQuery[] abstractFtsQueries = new AbstractFtsQuery[shouldQueries.size()];
			abstractFtsQueries = shouldQueries.toArray(abstractFtsQueries);
			query = query.should(abstractFtsQueries);
		}
		
		return bucket.query(new SearchQuery(searchIndex, query));
	}
	
	/*-
	 * <p>Find records by specified numeric values</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * SearchQueryResult result = CouchbaseFullTextService.findByNumberRange(bucket, INDEX_NAME, null, 5, false, 10, false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByNumberRange(bucket, INDEX_NAME, null, 5, false, 10, true);
	 * SearchQueryResult result = CouchbaseFullTextService.findByNumberRange(bucket, INDEX_NAME, "", 5, false, 10, false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByNumberRange(bucket, INDEX_NAME, "", 5, false, 10, true);
	 * SearchQueryResult result = CouchbaseFullTextService.findByNumberRange(bucket, INDEX_NAME, "attributes.attr", 5, false, 10, false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByNumberRange(bucket, INDEX_NAME, "attributes.attr", 5, true, 10, true);
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param field  the field name (ignore if null or empty)
	 * @param min  minimum numeric value to search
	 * @param minInclusive  whether to include min value in search results
	 * @param max  maximum numeric value to search
	 * @param maxInclusive  whether to include max value in search results
	 * @return  SearchQueryResult which contains document ids and hit locations (locations will be empty in this case)
	 */
	public static SearchQueryResult findByNumberRange(Bucket bucket, String searchIndex, String field, double min, boolean minInclusive, double max, boolean maxInclusive) {
		return bucket.query(new SearchQuery(searchIndex, SearchQuery.numericRange().min(min, minInclusive).max(max, maxInclusive).field(field)));
	}
	
	/*-
	 * <p>Find records by specified date values</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * SearchQueryResult result = CouchbaseFullTextService.findByDateRange(bucket, INDEX_NAME, "", "yyyy-MM-dd'T'hh:mm:ss.SSS'Z'", "2017-08-04T06:17:31.460Z", false, "", false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByDateRange(bucket, INDEX_NAME, "", "yyyy-MM-dd'T'hh:mm:ss.SSS'Z'", "2017-08-04T06:17:31.460Z", true, "", false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByDateRange(bucket, INDEX_NAME, "", "yyyy-MM-dd'T'hh:mm:ss.SSS'Z'", "", false, "2017-08-04T06:17:31.460Z", false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByDateRange(bucket, INDEX_NAME, "", "yyyy-MM-dd'T'hh:mm:ss.SSS'Z'", "", false, "2017-08-04T06:17:31.460Z", true);
	 * SearchQueryResult result = CouchbaseFullTextService.findByDateRange(bucket, INDEX_NAME, "", "yyyy-MM-dd'T'hh:mm:ss.SSS'Z'", "2017-08-03T06:17:31.460Z", false, "2017-08-04T06:17:31.460Z", false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByDateRange(bucket, INDEX_NAME, "", "yyyy-MM-dd'T'hh:mm:ss.SSS'Z'", "2017-08-03T06:17:31.460Z", true, "2017-08-04T06:17:31.460Z", false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByDateRange(bucket, INDEX_NAME, "", "yyyy-MM-dd'T'hh:mm:ss.SSS'Z'", "2017-08-03T06:17:31.460Z", false, "2017-08-04T06:17:31.460Z", false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByDateRange(bucket, INDEX_NAME, "createTime", "yyyy-MM-dd'T'hh:mm:ss.SSS'Z'", "2017-08-03T06:17:31.460Z", false, "2017-08-04T06:17:31.460Z", false);
	 * SearchQueryResult result = CouchbaseFullTextService.findByDateRange(bucket, INDEX_NAME, "attributes.activateTime", "yyyy-MM-dd'T'hh:mm:ss.SSS'Z'", "2017-08-03T06:17:31.460Z", false, "2017-08-04T06:17:31.460Z", false);
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param field  the field name (ignore if null or empty)
	 * @param dateFormat  the date format for parsing
	 * @param start  the start date for search
	 * @param startInclusive  whether to include start value in search results
	 * @param end  the end date for search
	 * @param endInclusive  whether to include end value in search results
	 * @return  SearchQueryResult which contains document ids and hit locations (locations will be empty in this case)
	 */
	public static SearchQueryResult findByDateRange(Bucket bucket, String searchIndex, String field, String dateFormat, String start, boolean startInclusive, String end, boolean endInclusive) {
		Validate.isTrue(StringUtils.isNotEmpty(start) || StringUtils.isNotEmpty(end), "Both %s and %s values cannot be null or empty. At least one value must be provided.", "start", "end");
		return bucket.query(new SearchQuery(searchIndex, SearchQuery.dateRange().start(start, startInclusive).end(end, endInclusive).field(field).dateTimeParser(dateFormat)));
	}
	
	/*-
	 * <p>Find records by specified query string</p>
	 * 
	 * <p>Examples:</p>
	 * <pre>
	 * SearchQueryResult result = CouchbaseFullTextService.findByQueryString(bucket, INDEX_NAME, "+janrainId:RegressionPatient -sample");
	 * SearchQueryResult result = CouchbaseFullTextService.findByQueryString(bucket, INDEX_NAME, "+janrainId:RegressionPatient attr");
	 * </pre>
	 * 
	 * @param bucket  the bucket name
	 * @param searchIndex  the FTS (Full-Text-Search) index name
	 * @param queryString  the user specified query string. Note that wildcards, regexp, and date range queries are not supported by this syntax
	 * @return  SearchQueryResult which contains document ids and hit locations
	 */
	public static SearchQueryResult findByQueryString(Bucket bucket, String searchIndex, String queryString) {
		Validate.notEmpty(queryString, "Value of %s cannot be null or empty", "queryString");
		return bucket.query(new SearchQuery(searchIndex, SearchQuery.queryString(queryString)));
	}
	
}
View original Gist ↗

The sample demonstrates the broader pattern of defining search behavior in the application while keeping the index responsible for making the relevant document fields searchable.

FTS vs. Structured Queries

FTS and structured database queries solve different problems. A structured query is typically better when the application knows the exact fields and conditions it needs to evaluate. FTS becomes useful when the requirement is search-oriented: relevance, tokenized text, partial matches, phrase matching, or other patterns that are cumbersome to express as conventional equality and range predicates.

Choosing between them should be driven by the access pattern rather than by the fact that both operate over the same JSON documents.

Takeaway

Full-Text Search introduces a search-specific indexing and query model alongside Couchbase's structured query capabilities. The key architectural decisions are what content to index, how to partition document types, and which queries genuinely require search semantics rather than standard structured filtering.