G
YouTube TikTok Games
Skip to content

About

fulltext-search-api

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

8 Commits

Folders and files

Repository files navigation

Full Text Search API

[![CI](https://github.com/rahulmoundekar/fulltext-search-api/actions/workflows/ci.yml/badge.svg)](https://github.com/rahulmoundekar/fulltext-search-api/actions/workflows/ci.yml)

A production-style Spring Boot REST API demonstrating PostgreSQL-native Full-Text Search, relevance ranking, fuzzy search, filtering, sorting, pagination, search suggestions, and database indexing.


๐Ÿ“Œ Project Overview

This project demonstrates how to build a powerful product search API using PostgreSQL Full-Text Search instead of introducing a separate search engine such as Elasticsearch.

The API supports:

  1. PostgreSQL Full-Text Search
  2. Fuzzy Search fallback

When a normal Full-Text Search doesn't return results, the API can fall back to PostgreSQL trigram-based fuzzy matching.

Main Search Flow

                    Search Request
                           |
                           v
                    Query Validation
                           |
                           v
                      PostgreSQL
                     Full-Text Search
                           |
                  +--------+--------+
                  |                 |
              Results            No Results
                  |                 |
                  |                 v
                  |           Fuzzy Search
                  |             pg_trgm
                  |                 |
                  +--------+--------+
                           |
                           v
                     Search Ranking
                           |
                           v
                        Filters
                           |
                           v
                         Sorting
                           |
                           v
                       Pagination
                           |
                           v
                    REST API Response

๐Ÿงญ Engineering Case Study

Concern Design decision Why it matters
Search engine choice PostgreSQL Full-Text Search first Keeps the stack simpler when database-native search fits
Typo tolerance pg_trgm + word_similarity() fallback Common misspellings still return useful candidates
Relevance Text score + rating + review popularity Ranking can reflect both lexical relevance and business signals
Performance GIN indexes + pagination Keeps retrieval efficient as the dataset grows

Full-text search architecture

๐Ÿš€ Features

  • PostgreSQL Full-Text Search
  • tsvector
  • websearch_to_tsquery()
  • ts_rank()
  • PostgreSQL pg_trgm
  • word_similarity()
  • Fuzzy search fallback
  • Search suggestions
  • Relevance-based ranking
  • Rating-based ranking
  • Review-count based popularity ranking
  • Brand filtering
  • Category filtering
  • Minimum price filtering
  • Maximum price filtering
  • Multiple sorting strategies
  • Pagination
  • Native PostgreSQL queries
  • Spring Data JPA projections
  • Flyway database migrations
  • PostgreSQL GIN indexes
  • Request validation
  • Global exception handling
  • Swagger/OpenAPI documentation

๐Ÿ›  Tech Stack

Technology Purpose
Java 21 Programming language
Spring Boot Application framework
Spring Web REST APIs
Spring Data JPA Persistence
PostgreSQL Database
PostgreSQL Full-Text Search Text search
PostgreSQL pg_trgm Fuzzy search
Flyway Database migration
Maven Build tool
Lombok Boilerplate reduction
Swagger / OpenAPI API documentation

๐Ÿ— Architecture

                         Client
                           |
                           v
                  ProductController
                           |
                           v
                   ProductService
                           |
             +-------------+-------------+
             |                           |
             v                           v
      Full-Text Search              Fuzzy Search
             |                           |
        tsvector                    pg_trgm
        tsquery                    word_similarity()
        ts_rank()                       |
             |                           |
             +-------------+-------------+
                           |
                           v
                    ProductRepository
                           |
                           v
                       PostgreSQL
                           |
                           v
                       products

๐Ÿ“‚ Project Structure

full-text-search-api/
โ”‚
โ”œโ”€โ”€ src/
โ”‚   โ”œโ”€โ”€ main/
โ”‚   โ”‚   โ”œโ”€โ”€ java/
โ”‚   โ”‚   โ”‚   โ””โ”€โ”€ com/rahul/api/
โ”‚   โ”‚   โ”‚       โ”‚
โ”‚   โ”‚   โ”‚       โ”œโ”€โ”€ config/
โ”‚   โ”‚   โ”‚       โ”‚   โ”œโ”€โ”€ OpenApiConfig.java
โ”‚   โ”‚   โ”‚       โ”‚   โ””โ”€โ”€ SearchConstants.java
โ”‚   โ”‚   โ”‚       โ”‚
โ”‚   โ”‚   โ”‚       โ”œโ”€โ”€ controller/
โ”‚   โ”‚   โ”‚       โ”‚   โ””โ”€โ”€ ProductController.java
โ”‚   โ”‚   โ”‚       โ”‚
โ”‚   โ”‚   โ”‚       โ”œโ”€โ”€ dto/
โ”‚   โ”‚   โ”‚       โ”‚   โ”œโ”€โ”€ ApiErrorResponse.java
โ”‚   โ”‚   โ”‚       โ”‚   โ”œโ”€โ”€ ProductRequest.java
โ”‚   โ”‚   โ”‚       โ”‚   โ”œโ”€โ”€ ProductResponse.java
โ”‚   โ”‚   โ”‚       โ”‚   โ”œโ”€โ”€ ProductSearchResponse.java
โ”‚   โ”‚   โ”‚       โ”‚   โ””โ”€โ”€ FuzzySearchResponse.java
โ”‚   โ”‚   โ”‚       โ”‚
โ”‚   โ”‚   โ”‚       โ”œโ”€โ”€ entity/
โ”‚   โ”‚   โ”‚       โ”‚   โ””โ”€โ”€ Product.java
โ”‚   โ”‚   โ”‚       โ”‚
โ”‚   โ”‚   โ”‚       โ”œโ”€โ”€ enums/
โ”‚   โ”‚   โ”‚       โ”‚   โ””โ”€โ”€ SearchSort.java
โ”‚   โ”‚   โ”‚       โ”‚
โ”‚   โ”‚   โ”‚       โ”œโ”€โ”€ exception/
โ”‚   โ”‚   โ”‚       โ”‚   โ”œโ”€โ”€ GlobalExceptionHandler.java
โ”‚   โ”‚   โ”‚       โ”‚   โ””โ”€โ”€ ProductNotFoundException.java
โ”‚   โ”‚   โ”‚       โ”‚
โ”‚   โ”‚   โ”‚       โ”œโ”€โ”€ repository/
โ”‚   โ”‚   โ”‚       โ”‚   โ”œโ”€โ”€ ProductRepository.java
โ”‚   โ”‚   โ”‚       โ”‚   โ””โ”€โ”€ projection/
โ”‚   โ”‚   โ”‚       โ”‚       โ”œโ”€โ”€ ProductSearchProjection.java
โ”‚   โ”‚   โ”‚       โ”‚       โ””โ”€โ”€ FuzzyProductProjection.java
โ”‚   โ”‚   โ”‚       โ”‚
โ”‚   โ”‚   โ”‚       โ””โ”€โ”€ service/
โ”‚   โ”‚   โ”‚           โ”œโ”€โ”€ ProductService.java
โ”‚   โ”‚   โ”‚           โ””โ”€โ”€ impl/
โ”‚   โ”‚   โ”‚               โ””โ”€โ”€ ProductServiceImpl.java
โ”‚   โ”‚   โ”‚
โ”‚   โ”‚   โ””โ”€โ”€ resources/
โ”‚   โ”‚       โ”œโ”€โ”€ db/
โ”‚   โ”‚       โ”‚   โ””โ”€โ”€ migration/
โ”‚   โ”‚       โ”‚       โ””โ”€โ”€ V1__create_products.sql
โ”‚   โ”‚       โ”‚
โ”‚   โ”‚       โ””โ”€โ”€ application.yml
โ”‚   โ”‚
โ”œโ”€โ”€ pom.xml
โ”œโ”€โ”€ README.md
โ””โ”€โ”€ .gitignore

๐Ÿ—„ Database

The project uses PostgreSQL.

Example database:

CREATE DATABASE fulltext_search;

๐Ÿ”Œ PostgreSQL Extensions

The project uses the PostgreSQL pg_trgm extension for fuzzy search.

CREATE EXTENSION IF NOT EXISTS pg_trgm;

Verify:

SELECT extname
FROM pg_extension
WHERE extname = 'pg_trgm';

Expected:

pg_trgm

๐Ÿ“Š Product Table

The application works with product fields such as:

id
title
description
brand
category
price
rating
review_count
search_vector
created_at
updated_at

The search_vector column stores the PostgreSQL Full-Text Search representation of searchable product data.


๐Ÿ”Ž PostgreSQL Full-Text Search

PostgreSQL Full-Text Search is the primary search mechanism.

The searchable data is represented using:

tsvector

and the user query is converted into:

tsquery

The search condition is:

p.search_vector @@ websearch_to_tsquery(
    'english',
    :query
)

The @@ operator determines whether the document matches the search query.


๐Ÿง  Why websearch_to_tsquery()?

The project uses:

websearch_to_tsquery()

instead of manually constructing a tsquery.

It provides search behavior closer to what users expect from a normal search box.

For example:

iphone pro

can be converted into a PostgreSQL search query without manually constructing PostgreSQL operators.


๐Ÿ“ˆ Search Ranking

Matching products are ranked using:

ts_rank()

Example:

SELECT
    p.id,
    p.title,
    ts_rank(
        p.search_vector,
        websearch_to_tsquery('english', :query)
    ) AS text_score
FROM products p
WHERE p.search_vector @@
      websearch_to_tsquery('english', :query)
ORDER BY text_score DESC;

โญ Custom Relevance Score

The API goes beyond simple text relevance.

The final ranking combines:

70% Text Relevance
20% Product Rating
10% Review Popularity

Formula:

(
    0.70 * ts_rank(
        p.search_vector,
        websearch_to_tsquery('english', :query)
    )
    +
    0.20 * COALESCE(p.rating, 0) / 5.0
    +
    0.10 * LEAST(
        LN(1 + COALESCE(p.review_count, 0)) / 10.0,
        1.0
    )
)

This allows the API to prioritize products that are both relevant and popular.


๐Ÿงฎ Why Combine Multiple Scores?

Consider two products:

Product A
Text relevance = 0.90
Rating = 4.8
Reviews = 1000

Product B
Text relevance = 0.92
Rating = 3.0
Reviews = 5

If we only use text relevance:

Product B > Product A

But with the combined ranking model:

Text relevance
+
Rating
+
Popularity

Product A can receive the better overall ranking.

This is closer to how real-world product search systems can be designed.


๐Ÿ”ค Fuzzy Search

Users don't always type the correct spelling.

Example:

Correct:
iphone

User enters:

iphne

Full-Text Search may not return a useful result.

The API therefore provides a fuzzy-search fallback.


๐Ÿงฉ PostgreSQL pg_trgm

The project uses:

pg_trgm

which provides trigram-based text similarity.

A trigram represents a sequence of three characters.

This allows PostgreSQL to calculate similarity between strings.


๐Ÿ” word_similarity()

The project uses:

word_similarity(
    LOWER(:query),
    LOWER(p.title)
)

Example:

SELECT
    title,
    word_similarity(
        LOWER('iphne'),
        LOWER(title)
    ) AS similarity
FROM products
ORDER BY similarity DESC;

This allows typo-tolerant matching.


โšก Trigram % Operator

The fuzzy search also uses:

LOWER(p.title) % LOWER(:query)

as a candidate filter.

The fuzzy condition is approximately:

WHERE
    LOWER(p.title) % LOWER(:query)

    AND word_similarity(
        LOWER(:query),
        LOWER(p.title)
    ) >= :threshold

The purpose is:

% operator
    โ†“
Find suitable candidates

word_similarity()
    โ†“
Calculate similarity

score
    โ†“
Rank results

๐Ÿ—‚ Database Indexes

Search performance depends heavily on indexes.

Full-Text Search Index

CREATE INDEX IF NOT EXISTS idx_products_search_vector
ON products
USING GIN (search_vector);

The GIN index allows PostgreSQL to efficiently search the tsvector column.


Fuzzy Search Index

CREATE INDEX IF NOT EXISTS idx_products_title_trgm
ON products
USING GIN (LOWER(title) gin_trgm_ops);

This index supports trigram-based searches.


๐Ÿ”„ Search Fallback Strategy

The main search endpoint follows this strategy:

                    Query
                      |
                      v
             Full-Text Search
                      |
              +-------+-------+
              |               |
            Found          Not Found
              |               |
              |               v
              |          Fuzzy Search
              |               |
              +-------+-------+
                      |
                      v
                    Rank
                      |
                      v
                   Filter
                      |
                      v
                    Sort
                      |
                      v
                  Paginate

This gives us the benefits of Full-Text Search while still handling common spelling mistakes.


๐Ÿ”Ž Search Filters

The search endpoint supports:

Brand

?brand=Apple

Category

?category=Smartphone

Minimum Price

?minPrice=50000

Maximum Price

?maxPrice=150000

Filters are applied to both:

Full-Text Search

and:

Fuzzy Search

This ensures that the fallback does not unexpectedly return products outside the user's requested filters.


โ†•๏ธ Sorting

The API supports:

RELEVANCE
PRICE_ASC
PRICE_DESC
RATING
NEWEST

Relevance

GET /api/v1/products/search?q=iphone&sort=RELEVANCE

Price Ascending

GET /api/v1/products/search?q=iphone&sort=PRICE_ASC

Price Descending

GET /api/v1/products/search?q=iphone&sort=PRICE_DESC

Rating

GET /api/v1/products/search?q=iphone&sort=RATING

Newest

GET /api/v1/products/search?q=iphone&sort=NEWEST

๐Ÿ“„ Pagination

The API uses Spring Data Pageable.

Example:

GET /api/v1/products/search?q=iphone&page=0&size=10

Example response:

{
  "content": [
    {
      "id": 1,
      "title": "Apple iPhone 17 Pro"
    }
  ],
  "totalElements": 9,
  "totalPages": 1,
  "size": 10,
  "number": 0
}

Fuzzy fallback also supports pagination using:

LIMIT :limit
OFFSET :offset

๐Ÿ”ฎ Search Suggestions

The API supports lightweight search suggestions.

Example:

GET /api/v1/products/suggestions?q=iph

Possible response:

[
  "Apple iPhone 17 Pro",
  "Apple iPhone 17 Pro Max"
]

Suggestions are useful for implementing autocomplete functionality in a frontend application.


๐Ÿ”Œ REST API Endpoints

Product APIs

Create Product

POST /api/v1/products

Request:

{
  "title": "Apple iPhone 17 Pro",
  "description": "Apple flagship smartphone",
  "brand": "Apple",
  "category": "Smartphone",
  "price": 129999
}

Get All Products

GET /api/v1/products

Get Product by ID

GET /api/v1/products/{id}

Example:

GET /api/v1/products/1

Update Product

PUT /api/v1/products/{id}

Example:

{
  "title": "Apple iPhone 17 Pro",
  "description": "Updated description",
  "brand": "Apple",
  "category": "Smartphone",
  "price": 125000
}

Delete Product

DELETE /api/v1/products/{id}

๐Ÿ”Ž Search API

GET /api/v1/products/search

Required parameter:

q

Optional parameters:

brand
category
minPrice
maxPrice
sort
page
size

Example:

GET /api/v1/products/search?q=iphone

Example with filters:

GET /api/v1/products/search?q=iphone&brand=Apple&category=Smartphone

Example with price range:

GET /api/v1/products/search?q=iphone&minPrice=50000&maxPrice=150000

Example with sorting and pagination:

GET /api/v1/products/search?q=iphone&sort=PRICE_ASC&page=0&size=10

๐Ÿงช Fuzzy Search API

GET /api/v1/products/fuzzy-search

Example:

GET /api/v1/products/fuzzy-search?q=iphne

Possible response:

[
  {
    "id": 1,
    "title": "Apple iPhone 17 Pro",
    "brand": "Apple",
    "category": "Smartphone",
    "price": 129999,
    "textScore": 0.5
  }
]

๐Ÿ’ก Search Examples

Exact Query

iphone

Expected matching products:

Apple iPhone 17 Pro
Apple iPhone 17 Pro Max

Typo Query

iphne

The Full-Text Search may not return results.

The API can then perform fuzzy matching:

iphne
  |
  v
pg_trgm
  |
  v
iPhone

โš ๏ธ Validation

The API validates search queries.

For example:

GET /api/v1/products/search?q=

returns:

{
  "status": 400,
  "error": "Bad Request",
  "message": "Search query must not be empty"
}

Minimum and maximum query lengths are controlled centrally through SearchConstants.


โŒ Error Handling

The API uses a global exception handler.

@RestControllerAdvice

The following cases are handled:

  • Invalid search query
  • Invalid price range
  • Product not found
  • Request validation errors
  • Unexpected server errors

Example:

{
  "timestamp": "2026-08-19T20:00:00",
  "status": 404,
  "error": "Not Found",
  "message": "Product not found with id: 999",
  "path": "/api/v1/products/999"
}

๐Ÿ›ซ Flyway

Database schema changes are managed using Flyway.

Migration files are located under:

src/main/resources/db/migration/

Example:

V1__create_products.sql

Flyway automatically executes pending migrations during application startup.


โš™๏ธ Configuration

Example application.yml:

spring:
  datasource:
    url: jdbc:postgresql://localhost:5432/fulltext_search
    username: ${DB_USERNAME:postgres}
    password: ${DB_PASSWORD:postgres}

  jpa:
    hibernate:
      ddl-auto: validate

  flyway:
    enabled: true

Use environment variables for real credentials.

Do not commit production passwords to GitHub.


โ–ถ๏ธ Running the Application

Prerequisites

Install:

  • Java 21
  • Maven
  • PostgreSQL

Verify Java:

java -version

Verify Maven:

mvn -version

1. Create Database

CREATE DATABASE fulltext_search;

2. Enable PostgreSQL Extension

CREATE EXTENSION IF NOT EXISTS pg_trgm;

3. Configure Database

Set:

DB_USERNAME
DB_PASSWORD

or configure your local application.yml.


4. Start Application

mvn spring-boot:run

Application:

http://localhost:8080

๐Ÿ“š Swagger / OpenAPI

Swagger UI:

http://localhost:8080/swagger-ui/index.html

OpenAPI specification:

http://localhost:8080/v3/api-docs

Swagger provides an interactive way to execute and test the APIs.


๐Ÿง  Important Concepts Learned

PostgreSQL

  • Full-Text Search
  • tsvector
  • tsquery
  • websearch_to_tsquery
  • ts_rank
  • pg_trgm
  • word_similarity
  • % trigram operator
  • GIN indexes
  • Native SQL
  • EXPLAIN ANALYZE

Spring Boot

  • REST controllers
  • Service layer
  • Repository layer
  • Spring Data JPA
  • Native queries
  • Interface projections
  • Pageable
  • DTOs
  • Validation
  • Global exception handling

API Design

  • Search filters
  • Sorting
  • Pagination
  • Search fallback
  • Relevance ranking
  • Error responses

๐Ÿ”ฌ Performance Considerations

The project uses PostgreSQL indexes specifically designed for the search workload.

Full-Text Search:

GIN(search_vector)

Fuzzy search:

GIN(LOWER(title) gin_trgm_ops)

Use:

EXPLAIN ANALYZE

to verify query plans and index usage.

Example:

EXPLAIN ANALYZE
SELECT
    id,
    title
FROM products
WHERE search_vector @@
      websearch_to_tsquery('english', 'iphone');

๐Ÿ†š PostgreSQL FTS vs LIKE

A simple search might use:

WHERE LOWER(title) LIKE '%iphone%'

However, Full-Text Search provides features such as:

  • Tokenization
  • Normalization
  • Ranking
  • Language-aware search
  • Search operators
  • GIN indexing

Therefore:

LIKE

is useful for simple substring matching, while:

Full-Text Search

is better suited for richer text-search requirements.


๐Ÿ†š PostgreSQL FTS vs Elasticsearch

This project intentionally uses PostgreSQL.

PostgreSQL advantages

  • Existing database infrastructure
  • No additional search cluster
  • ACID transactions
  • Simple deployment
  • Native SQL
  • GIN indexes
  • Full-Text Search
  • Trigram fuzzy search

Elasticsearch/OpenSearch may be preferred when

  • Search volume is extremely large
  • Advanced search capabilities are required
  • Distributed search is required
  • Complex relevance models are required
  • Search analytics are extensive

The appropriate solution depends on the application's scale and requirements.


๐ŸŽฏ Interview Explanation

A concise explanation of this project:

I implemented a product search API using PostgreSQL Full-Text Search. Product data is indexed into a tsvector column and queries are generated using websearch_to_tsquery(). Results are ranked using ts_rank() and a custom score that combines text relevance, product rating and review popularity. If Full-Text Search doesn't find results, the API falls back to PostgreSQL pg_trgm fuzzy search using word_similarity(). I also implemented filtering, sorting, pagination, GIN indexes, native queries and Swagger documentation.


๐Ÿ’ฌ Interview Questions

1. Why use PostgreSQL Full-Text Search?

To provide efficient text searching and ranking directly inside PostgreSQL without requiring a separate search engine.

2. What is tsvector?

A normalized representation of searchable text used by PostgreSQL Full-Text Search.

3. What is tsquery?

A PostgreSQL representation of a search query used to match against a tsvector.

4. What does @@ do?

It checks whether a tsvector matches a tsquery.

5. What is ts_rank()?

It calculates the relevance score of a matching document.

6. Why use websearch_to_tsquery()?

It provides search syntax that is more natural for user-entered search strings.

7. What is pg_trgm?

A PostgreSQL extension providing trigram-based similarity operations and indexes.

8. Why use word_similarity()?

To find text that is similar even when the user makes spelling mistakes.

9. Why use a GIN index?

GIN indexes efficiently support PostgreSQL Full-Text Search and trigram-based searches.

10. Why use native SQL?

PostgreSQL-specific features such as:

ts_rank()
websearch_to_tsquery()
word_similarity()

are database-specific and are easier to express using native SQL.

11. Why use projections?

Search queries often return calculated values such as:

textScore
score
titleHighlight
descriptionHighlight

Interface projections allow the application to retrieve only the required columns instead of loading complete entities.

12. How does fuzzy fallback work?

The API first performs Full-Text Search.

If there are no results, it executes trigram-based fuzzy search and returns similar products.


๐Ÿ”ฎ Future Improvements

Potential future enhancements include:

  • Search title + description + brand
  • PostgreSQL dictionaries
  • Synonym support
  • Search highlighting
  • Advanced autocomplete
  • Search analytics
  • Popular search tracking
  • Redis caching
  • Search result caching
  • Elasticsearch/OpenSearch migration for very large workloads
  • Advanced ranking models
  • Multi-language search
  • Search typo correction
  • Search history

๐Ÿ” Security

This project focuses primarily on PostgreSQL search concepts.

Authentication and authorization can be integrated using Spring Security/JWT when required.


๐Ÿ“ Project Status

โœ… Product CRUD
โœ… PostgreSQL database
โœ… Flyway migration
โœ… Full-Text Search
โœ… tsvector
โœ… tsquery
โœ… websearch_to_tsquery()
โœ… ts_rank()
โœ… Custom relevance score
โœ… Brand filtering
โœ… Category filtering
โœ… Price filtering
โœ… Pagination
โœ… Sorting
โœ… Fuzzy search
โœ… pg_trgm
โœ… word_similarity()
โœ… Search suggestions
โœ… GIN indexes
โœ… Validation
โœ… Global exception handling
โœ… Swagger/OpenAPI
โœ… GitHub README

๐Ÿ“Œ Learning Outcome

After completing this project, you should understand how to implement a production-style search API using PostgreSQL without immediately introducing a dedicated search engine.

The most important concepts demonstrated are:

PostgreSQL Full-Text Search
        +
Relevance Ranking
        +
Fuzzy Search
        +
Database Indexing
        +
Filtering
        +
Sorting
        +
Pagination
        +
Spring Boot REST API

๐Ÿ‘จโ€๐Ÿ’ป Author

Rahul Moundekar

Java Backend Developer

Technologies:

Java
Spring Boot
Microservices
REST APIs
PostgreSQL
JPA
Docker
Cloud

๐Ÿ“„ License

This project is intended for learning, portfolio and interview demonstration purposes.

About

fulltext-search-api

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages