Search

QWC2 can be configured to use arbitrary custom search providers. In addition, the qwc-fulltext-search-service provided by the qwc-services ecosystem can be used.

Adding search providers

Search providers can be defined as follows:

  • Built-in, defined in js/SearchProviders.js. This file is structured as follows:
export const SearchProviders = {
    <providerkey1>: <ProviderDefinition1>,
    <providerkey2>: <ProviderDefinition2>,
    ...
};

Built-in search providers are compiled into the application bundle and avoid the need for an extra resource to be loaded on application startup. The downside is that you need to rebuild QWC2 to add/modify search providers.

  • As resource, defined in static/assets/searchProviders.js. This file is structured as follows:
window.QWC2SearchProviders = {
    <providerkey1>: <ProviderDefinition1>,
    <providerkey2>: <ProviderDefinition2>,
    ...
};

This script file needs to be loaded explicitly by index.html via

<script type="text/javascript" src="assets/searchProviders.js" ></script>

The format of ProviderDefinition is

{
  label: "<human readable provider name>", // OR
  labelmsgid: "<translation message ID for human readable provider name>",
  onSearch: function(searchText, searchParams, callback, axios) => {
    const results = []; // See below
    /* Populate results... */
    callback({results: results});
  },
  getResultGeometry: function(resultItem, callback, axios) => {
    /* Retreive geometry... */
    // resultItem is a search result entry as returned by onSearch, which provides the context for retreiving the geometry
    const geometry = "<wktString>";
    // or
    const geometry = {<GeoJSON geometry>};

    const crs = "EPSG:XXXX";
    const hidemarker = <boolean>; // Whether to suppress displaying a search marker on top of the search geometry
    callback({geometry: geometry, crs: crs, hidemarker: hidemarker});
  },
  handlesGeomFilter: <boolean>; // Hint whether provider will completely filter the results on provider side and that no client-side filtering is necessary
}

Notes:

  • The format of searchParams is
{
  displaycrs: "EPSG:XXXX", // Currently selected mouse coordinate display CRS
  mapcrs: "EPSG:XXXX", // The current map CRS
  lang: "<code>", // The current application language, i.e. en-US or en
  cfgParams: <params> // Additional parameters passed in the theme search provider configuration, see below
  filterBBox: <[xmin, ymîn, xmax, ymax]|null> // A filter bbox, in mapcrs, the search component may pass to the provider to narrow down the results
  filterPoly: <[[x0, y0], [x1, y1], ....]> // A filter polygon, in mapcrs, the search component may pass to the provider to narrow down the results
}

  • axios is passed for convenience so that providers can use the compiled-in axios library for network requests.

  • The format of the results list returned by onSearch is as follows:

results = [
  {
    id: "<categoryid>",                   // Unique category ID
    title: "<display_title>",             // Text to display as group title in the search results
    priority: priority_nr,                // Optional: search result group priority. Groups with higher priority are displayed first in the list.
    items: [
      {                                   // Location search result:
        type: SearchResultType.PLACE,     // Specifies that this is a location search result
        id: "<itemId">,                   // Unique item ID
        text: "<display text>",           // Text to display as search result
        label: "<map marker text>",       // Optional, text to show next to the position marker on the map instead of `text`
        x: x,                             // X coordinate of result
        y: y,                             // Y coordinate of result
        crs: crs,                         // CRS of result coordinates and bbox
        bbox: [xmin, ymin, xmax, ymax],   // Bounding box of result (if non-empty, map will zoom to this extent when selecting result)
        geometry: <GeoJSON geometry>      // Optional, result geometry. Note: geometries may also be fetched separately via getResultGeometry.
      },
      {                                    // Theme layer search result (advanced):
        type: SearchResultType.THEMELAYER, // Specifies that this is a theme layer search result
        id: "<itemId">,                    // Unique item ID
        text: "<display text>",            // Text to display as search result
        layer: {<Layer definition>}        // Layer definition, in the same format as a "sublayers" entry in themes.json.
      }
    ]
  },
  {
    ...
  }
]

Consult js/SearchProviders.js and static/assets/searchProviders.js for full examples.

Filtering

When using the SearchBox search component with allowSearchFilters: true passed in the TopBar searchOptions, you a filter menu will be displayed allowing to restrict the search area.

If the provider does not fully handle the filter geometry internally (handlesGeomFilter != true), client-side filtering will be performed as follows:

  1. Polygon intersection test if the result has a geometry field with a Polygon geometry
  2. Polygon intersection test if the result has a bbox field
  3. Point-in-polygon test using the results x and y point coordinates

You can also set a predefined list of filter areas by setting searchFilterRegions in config.json (or per-theme in themesConfig.json) as follows:

"searchFilterRegions": [
  {
    "name": "<Group name>",
    "items": [
      {
        "name": "<Name>",
        "crs": "<EPSG:XXXX>",
        "coordinates": [[x0, y0], [x1, y1], ...]
      },
      ...
    ]
  },
  ...
]
 ```

## Configuring theme search providers

For each theme item in `themesConfig.json`, you can define a list of search providers to enable for the theme as follows:
```json
...
searchProviders: [
  "<providerkey1>",             // Simple form
  {                             // Provider with custom params
    "provider": "<providerkey2>",
    "params": {
      ...                       // Arbitrary params passed to the provider `onSearch` function as `searchParams.cfgParams`
    }
  },
  {                             // Fulltext search configuration using qwc-fulltext-search-service
    "provider":"solr",          // Identifier for solr search provider
    "default":[<default terms>] // Default search terms, concatenated with additional search terms from visible theme layers
  }
],
...

Note: The qwc2-demo-app (also used by the qwc-map-viewer-demo docker image) includes three providers by default: coordinates, nominatim (OpenStreetMap location search), and qgis (see below).

The QGIS feature search relies on WMS GetFeatureInfo with the FILTER parameter to search features of layers which are part of the theme WMS. It is enabled via the qgis search provider, which is part of the qwc2-demo-app.

Note: Make sure the QGIS Project is configured to return geometries with the feature info responses (Project → Properties → QGIS Server → Add geometry to feature response).

In it's simples form, you can configure the theme search provider entry as follows:

  {
    "provider": "qgis",
    "params": {
      "title": "<search name>",
      "expression": {
        "<layername1>": "<expression>",
        "<layername2>": "<expression>"
      }
    }
  }

where expression is a WMS GetFeatureInfo FILTER expression, for example "\"name\" ILIKE '%$TEXT$%'". $TEXT$ will be replaced by the search text entered by the user, name corresponds to a field name of the specified layer.

A more complex form, useable through the FeatureSearch plugin, allows defining a field configuration for multiple input fields. A full example is as follows:

  {
    "provider": "qgis",
    "params": {
      "title": "Person search",
      "expression": {
        "persons": "\"name\" ILIKE '%$NAME$%' AND \"age\" >= $AGE$ AND \"gender\" = '$GENDER$'"
      },
      "fields": {
        "NAME": {"label": "Name", "type": "text"},
        "AGE": {"label": "Min. age", "type": "number", "options": {"min": 0}},
        "GENDER": {"label": "Gender", "type": "select", "options": [{"value": "f", "label": "Female"}, {"value": "m", "label": "Male"}]}
      }
    }
  }

Here, each field will provide a value which is substituted in the expression. Any HTML Input type is supported (i.e. text, number, range, ...), with options depending on the input type. In addition, the select field type is supported to display a ComboBox, with the entries provided as options as in the example above. It is also possible to pass a flat list as options, i.e. ["Female", "Male"] if the value is equal to the label.

Note: qgis provider searches are exposed to the search field only if no fields are specified (i.e. single input search). The FeatureSearch plugin on the other hand will list all qgis provider searches.

In addition to the configuration described above, you can specify these additional parameters in params:

  • featureCount: A number, passed as feature_count to the GetFeatureInfo request to control the maximum number of returned features. If not specified, defaults to 100.
  • resultTitle: A format string for the result title. Allowed placeholders are: {layername} for the layer name and {<fieldname>} for the value of fieldname. If not the layer name followed by the feature displayfield will be shown.
  • description: An arbitrary descriptive text which will be displayed above the search fields in the FeatureSearch plugin.
  • default: Whether the search is selected by default when opening the FeatureSearch plugin.
  • group: A group name, used to group the searches in the FeatureSearch selection combobox.

Configuring the fulltext search service

The qwc-fulltet-search-service provides facetted fullsearch text search, with one of the following backends:

  • Postgres Trigram
  • Apache Solr

A facet references a searchable dataset. The configuration of the fulltext search service and available search facets can be found in tenantConfig.json:

{
    "name": "search",
    "config": {
        "search_backend": "<solr|trgm>",
        "word_split_re": "[\\s,.:;\"]+",
        "search_result_limit": 50,
        "db_url": "postgresql:///?service=qwc_geodb",
        // trgm specific configuration, see below
        "trgm_feature_query": "<see below>",
        "trgm_layer_query": "<see below>",
        "trgm_similarity_threshold": "0.3"
        // solr specific configuration, see below
        "solr_service_url": "http://localhost:8983/solr/gdi/select",
        "search_result_sort": "score desc, sort asc",
    },
    "resources": {
        "facets": [
            {
                "name": "<facet name>",
                "filter_word": "<filter word>",
                "table_name": "<schema.tablename>",
                "geometry_column": "<geometry column name>",
                "search_id_col": "<id column name>"
            },
            ...
        ]
    }
}
  • The search_backend specifies the search backend to use, either solr or trgm. Default: solr.
  • The db_url specifies the DB which contains the search index (searched either by solr or by the specified trgm queries).
  • The word_split_re specifies the regular expression which is used to split the search string into single words. Default: [\\s,.:;\"]+.
  • search_result_limit specifies the maximum number of feature results returned by a search. Default: 50.

The facets describe a searchable dataset and are referenced by the search index:

  • name specifies the facet identifier.
  • filter_word is a short (human readable) name which appears as result category in the search results (i.e. Address).
  • table_name specifies the table containing the features referenced by the search index (in the format schema.table_name).
  • geometry_column specifies the name of the geometry column in this table.
  • search_id_col specifies the name of the id column in this table. If unset, field from search filter expression is used.

Fulltext search with Trigram backend

To configure a fulltext search with the trigram backend, set search_backend to trgm and specify a trgm_feature_query and optionally a trgm_layer_query. The feature and layer query SQL can contain following placeholders:

  • :term: The full search text
  • :terms: A list of search text words (i.e. the full search text split by whitespace).
  • :thres: The trigram similarity treshold value (note that the service will also separately execute SET pg_trgm.similarity_threshold = <value>)

The trgm_feature_query must return the following fields:

  • display: The label to display in the search results.
  • facet_id: The facet name (as configured in resources => facets).
  • id_field_name: The name of the identifier field in the table referenced by the facet.
  • feature_id: The feature identifier through which to locate the feature in table referenced by the facet.
  • bbox: The feature bounding box, as a[xmin,ymin,xmax,ymax] string.
  • srid: The SRID of the bbox coordinates (i.e. 3857).

Example:

SELECT display, facet_id, id_field_name, feature_id, bbox, srid, similarity(suchbegriffe, :term) sml
FROM public.search_index WHERE searchterms % :term OR searchterms ILIKE '%' || :term || '%' ORDER BY sml DESC;",

The trgm_layer_query must return the following fields:

  • display: The label to display in the search results.
  • dataproduct_id: The id of the dataproduct.
  • has_info: Whether an abstract is available for the dataproduct.
  • sublayers: A JSON stringified array of the shape [{"ident": "<dataproduct_id>", "display": "<display>", "dset_info": true}, ...], or NULL if no sublayers exist.

Note: The layer query relies on an additional service, configured as dataproductServiceUrl in the viewer config.json, which resolves the dataproduct_id to a QWC theme sublayer object, like the sogis-dataproduct-service.

Note: Set FLASK_DEBUG=1 as environment variable for the search service to see additional logging information.

Fulltext search with Solr backend

To use the solr backend, you need to run a solr search service and point solr_service_url to the corresponding URL. You can find the solr documentation at https://lucene.apache.org/solr/guide/8_0/.

Next, create search XML configuration files in volumes/solr/configsets/gdi/conf/. The name of the file can be chosen freely. Example:

<dataConfig>
    <dataSource
        driver="org.postgresql.Driver"
        url="jdbc:postgresql://{DB_HOST}:{DB_PORT}/{DB_NAME}"
        user="{DB_USER}"
        password="{DB_PASSWORD}"
    />
    <document>
        <entity name="{FACET_NAME}" query="
            WITH index_base AS (
                /* ==== Base query for search index ==== */
                SELECT
                    '{FACET_NAME}'::text AS subclass,
                    {PRIMARY_KEY} AS id_in_class,
                    '{PRIMARY_KEY}' AS id_name,
                    'str:{SEARCH_FIELD_IS_STRING}' AS search_field_str,
                    {DISPLAYTEXT} AS displaytext,
                    {SEARCH_FIELD_1} AS search_part_1,
                    {GEOMETRY_FIELD} AS geom
                FROM {SCHEMA}.{SEARCH_TABLE_NAME}
                /* ===================================== */
            )
            SELECT
                (array_to_json(array_append(ARRAY[subclass::text], id_in_class::text)))::text AS id,
                displaytext AS display,
                search_part_1 AS search_1_stem,
                search_part_1 AS sort,
                subclass AS facet,
                'default' AS tenant,
                (array_to_json(array_append(ARRAY[id_name::text], search_field_str::text)))::text AS idfield_meta,
                (st_asgeojson(st_envelope(geom), 0, 1)::json -> 'bbox')::text AS bbox,
                id_name,
                geom
            FROM index_base">
        </entity>
    </document>
</dataConfig>

The next table shows how the values need to be defined:

Name Definition Example
DB_HOST Database hostname qwc-postgis
DB_NAME Database name qwc_demo
DB_PORT Database port number 5432
DB_USER Database username qwc_service
DB_PASSWORD Password for the specified database user qwc_service
FACET_NAME Name of the search facet fluesse_search
PRIMARY_KEY Primary key name of the table that is used in the search query ogc_fid
SEARCH_FIELD_IS_STRING Definition, if search field is string (y) or not (n). If not, it's interpreted as integer. n
DISPLAYTEXT Displaytext that will be shown by the QWC2 when a match was found name_long
SEARCH_FIELD_1 Table field that will be used by the search name_long
GEOMETRY_FIELD Name of the geometry column of the search table wkb_geometry
SCHEMA Search table schema qwc_geodb
SEARCH_TABLE_NAME Search table name fluesse

Hint: For a less complex configuration file, of course it is also possible to define the query within a VIEW definition within the database. In this case just provide the query within the facet configuration like:

<entity name="{FACET_NAME}" query="
    SELECT id,
    display,
    search_1_stem,
    sort,
    facet,
    tenant,
    idfield_meta,
    bbox
    id_name,
    geom
    FROM index_base">
</entity>

Note: In the case of several searches sharing the same database connection, all searche queries can be written to the same XML file. Each search corresponds to exactly one <entity> tag in the XML file.

After the configuration file has been created, the search must be registered in solr. In the volumes/solr/configsets/gdi/conf/solrconfig.xml file you have to look for <!-- SearchHandler and add the following configuration

<requestHandler name="/FACET_NAME" class="solr.DataImportHandler">
    <lst name="defaults">
        <str name="config">NAME_OF_THE_CONFIGURATION_FILE.xml</str>
    </lst>
</requestHandler>

Finally, the solr index has to be generated:

rm -rf volumes/solr/data/*
docker compose restart qwc-solr
curl 'http://localhost:8983/solr/gdi/FACET_NAME?command=full-import'

Configuring the search for a theme

To use a fulltext search in a theme, configure a fulltext search provider in themesConfig.json as follows:

"searchProviders": [
    {
        "provider": "fulltext",
        "default": [<FACET_NAME>],
        "layers": {
            "<layer_name>": "<FACET_NAME>"
        }
    }
]

Where:

  • default lists the search facets enabled by default.
  • layers providides a mapping of facets which are enabled whenever the theme layer <layer_name> is visible on the map.

Next, create Search facet resources in the Admin GUI for the desired facet names.

  • To manage layer search permissions, you can create a Search facet with name dataproduct.
  • You can create a wildcard Search facet resource by setting the name to *. This is useful to assign permissions for all available facets with one single resource.

Finally, create permissions for the newly created resources and regenerate the service configuration.