search.integration

Classes

TolerantTransport

A transport class that is less eager to rejoin connections when there's

ElasticsearchApp

Provides elasticsearch integration for

Functions

is_5xx_error(→ bool)

process_indexer_tween_factory(...)

Module Contents

class search.integration.TolerantTransport(*args: Any, **kwargs: Any)[source]

Bases: elasticsearch.Transport

A transport class that is less eager to rejoin connections when there’s a failure. Additionally logs all Elasticsearch transport errors in one location.

failure_time: datetime | None[source]
failures = 0[source]
property skip_request: bool[source]

Returns True if the request should be skipped.

property seconds_remaining: int[source]

Returns the seconds remaining until the next try or 0.

For each failure we wait an additional 10s (10s, then 20s, 30s, etc), up to a maximum of 300s (5 minutes).

perform_request(*args: Any, **kwargs: Any) → Any[source]

Perform the actual request. Retrieve a connection from the connection pool, pass all the information to it’s perform_request method and return the data.

If an exception was raised, mark the connection as failed and retry (up to max_retries times).

If the operation was successful and the connection used was previously marked as dead, mark it as live, resetting it’s failure count.

Parameters:

  • method – HTTP method to use

  • url – absolute url (without host) to target

  • headers – dictionary of headers, will be handed over to the underlying Connection class

  • params – dictionary of query parameters, will be handed over to the underlying Connection class for serialization

  • body – body of the request, will be serialized using serializer and passed to the connection

search.integration.is_5xx_error(error: elasticsearch.TransportError) → bool[source]
class search.integration.ElasticsearchApp[source]

Bases: morepath.App

Provides elasticsearch integration for onegov.core.framework.Framework based applications.

The application must be connected to a database.

Usage:

from onegov.core import Framework

class MyApp(Framework, ESIntegration):
    pass
schema: str[source]
es_client: elasticsearch.Elasticsearch | None[source]

Configures the elasticsearch client, leaving it as a property on the class:

app.es_client

The following configuration options are accepted:

Enable_elasticsearch:

If True, elasticsearch is enabled (defaults to True).

Elasticsearch_hosts:

A list of elasticsearch clusters, including username, password, protocol and port.

For example: https://user:secret@localhost:443

By default the client connects to the localhost on port 9200 (the default), and on port 19200 (the default of boxen).

At least one host in the list of servers must be up at startup.

Elasticsearch_may_queue_size:

The maximum queue size reserved for documents to be indexed. This queue is filling up if the elasticsearch cluster cannot be reached.

Once the queue is full, warnings are emitted.

Defaults to 10’000

Elasticsearch_verify_certs:

If true, the elasticsearch client verifies the certificates of the ssl connection. Defaults to true. Do not disable, unless you are in testing!

Elasticsearch_languages:
The languages supported by onegov.search. Defaults to:

  • en

  • de

  • fr

es_configure_client(usage: Literal['default', 'reindex'] = 'default') → None[source]

Returns a search scoped to the current application, with the given languages, types and private documents excluded by default.

es_indices(languages: Iterable[str] = '*', types: Iterable[str] = '*') → str[source]
es_search_by_request(request: onegov.core.request.CoreRequest, types: Iterable[str] = '*', explain: bool = False, limit_to_request_language: bool = False) → onegov.search.Search[source]

Takes the current CoreRequest and returns an elastic search scoped to the current application, the requests language and it’s access rights.

es_suggestions(query: str, languages: Iterable[str] = '*', types: Iterable[str] = '*', include_private: bool = False) → tuple[str, Ellipsis][source]

Returns suggestions for the given query.

es_suggestions_by_request(request: onegov.core.request.CoreRequest, query: str, types: Iterable[str] = '*', limit_to_request_language: bool = False) → tuple[str, Ellipsis][source]

Returns suggestions for the given query, scoped to the language and the login status of the given requst.

Returns True if the given request is allowed to access private search results. By default every logged in user has access to those.

This method may be overwritten if this is not desired.

get_searchable_models() → list[type[onegov.search.mixins.Searchable]][source]
es_perform_reindex(fail: bool = False) → None[source]

Re-indexes all content.

This is a heavy operation and should be run with consideration.

By default, all exceptions during reindex are silently ignored.

search.integration.process_indexer_tween_factory(app: ElasticsearchApp, handler: Callable[[CoreRequest], Response]) → Callable[[CoreRequest], Response][source]