ievv_elasticsearch — Thin wrapper around pyelasticsearch¶
The ievv_elasticsearch
module is designed to just make it
a small bit easier to work with elasticsearch in Django than to just
use pyelasticsearch.
Features¶
- Makes
pyelasticsearch
easy to use in Django. - Very thin wrapper around
pyelasticsearch
. This means that you use the elasticsearch REST API almost directly with just some pythonic glue. - Automatic indexing of data store data is decoupled from the search/get API.
- Automatic indexing of data store data works with any data store. Our examples use Django ORM, but you can just as easily use a NOSQL database like MongoDB or an XML database. The only difference is the code you provide to convert your data store data into ElasticSearch JSON compatible data structures.
- Small Django/python helpers that enhances the
pyelasticsearch
API. - Shortcuts and solutions that makes unit testing easy.
Why not use Haystack?¶
Haystack is an awesome library for full text document search in a
search engine independent manner. But if you want to make full
use of ElasticSearch as more than just a document search index,
and use it for analytics, document caching and a general purpose
nosql data store, haystack just adds an extra layer of complexity
that you have to work around. This is, of course, use-case dependent,
and many use cases will probably be better served by a combination of
ievv_elasticsearch
and Haystack.
Note
If considering combining Haystack with ievv_elasticsearch
, you
should know that ievv_elasticsearch
has loose coupling between
index definition and querying. Indexe definitions in ievv_elasticsearch
are
only used to make it easy to sync the backend data store into an elasticseach
index. If you define the search indexes in haystack, you can still use
the ievv_opensource.ievv_elasticsearch.search
API, you just ignore
ievv_opensource.ievv_elasticsearch.autoindex
.
Getting started¶
You only need the following to get started:
- ElasticSearch server.
- Configure
IEVV_ELASTICSEARCH_URL
with the URL of the server.
Then you can start using the ievv_opensource.ievv_elasticsearch.search.Connection
API.
Setup for unit-testing and development¶
First, copy not_for_deploy/elasticsearch.develop.yml
and not_for_deploy/elasticsearch.unittest.yml
into your own project.
In your test settings, add:
IEVV_ELASTICSEARCH_TESTURL = 'http://localhost:9251'
IEVV_ELASTICSEARCH_TESTMODE = True
IEVV_ELASTICSEARCH_AUTOREFRESH_AFTER_INDEXING = True
In your develop settings, add:
IEVV_ELASTICSEARCH_URL = 'http://localhost:9252'
When this is configured, you can run elasticsearch with ievv devrun — All your development servers in one command if
you add the following to IEVVTASKS_DEVRUN_RUNNABLES
:
IEVVTASKS_DEVRUN_RUNNABLES = {
'default': ievvdevrun.config.RunnableThreadList(
# ...
ievvdevrun.runnables.elasticsearch.RunnableThread(configpath='not_for_deploy/elasticsearch.unittest.yml'),
ievvdevrun.runnables.elasticsearch.RunnableThread(configpath='not_for_deploy/elasticsearch.develop.yml'),
),
]
(the paths assumes you put the configfiles in the not_for_deploy/
directory in your project).
Automatically update the search indexes¶
Unless you use ElasticSearch as the primary data source, you will most likely want an easy method of: 1. Update the search index when data in the data store changes. 2. Rebuild the search index from the data in the data store.
This is solved by:
- Define a
ievv_opensource.ievv_elasticsearch.autoindex.AbstractIndex
. Theievv_elasticsearch
convention is to put search indexes inyourapp.elasticsearch_autoindexes
. - Register the index class in
ievv_opensource.ievv_elasticsearch.autoindex.Registry
. - Optionally override
ievv_opensource.ievv_elasticsearch.autoindex.AbstractIndex.register_index_update_triggers()
. and register triggers that react to data store changes and trigger re-indexing.
search API¶
ievv_opensource.ievv_elasticsearch.search.Connection |
Singleton wrapper around pyelasticsearch.ElasticSearch . |
ievv_opensource.ievv_elasticsearch.search.SearchResultWrapper |
An efficient wrapper around the data returned by pyelasticsearch.ElasticSearch.search() . |
ievv_opensource.ievv_elasticsearch.search.SearchResultItem |
Wrapper around a single dictionary in the hits.hits list of the result returned by pyelasticsearch.ElasticSearch.search() . |
ievv_opensource.ievv_elasticsearch.search.Paginator |
Paginator for ievv_opensource.ievv_elasticsearch.search.SearchResultWrapper . |
-
class
ievv_opensource.ievv_elasticsearch.search.
IevvElasticSearch
(urls='http://localhost', timeout=60, max_retries=0, port=9200, username=None, password=None, ca_certs='/home/docs/checkouts/readthedocs.org/user_builds/ievv-opensource/envs/stable/lib/python3.5/site-packages/certifi/cacert.pem', client_cert=None)[source]¶ Bases:
pyelasticsearch.client.ElasticSearch
Parameters: - urls – A URL or iterable of URLs of ES nodes. These can be full
URLs with port numbers, like
http://elasticsearch.example.com:9200
, or you can pass the port separately using theport
kwarg. To do HTTP basic authentication, you can use RFC-2617-style URLs likehttp://someuser:somepassword@example.com:9200
or the separateusername
andpassword
kwargs below. - timeout – Number of seconds to wait for each request before raising Timeout
- max_retries – How many other servers to try, in series, after a request times out or a connection fails
- username – Authentication username to send via HTTP basic auth
- password – Password to use in HTTP basic auth. If a username and password are embedded in a URL, those are favored.
- port – The default port to connect on, for URLs that don’t include an explicit port
- ca_certs – A path to a bundle of CA certificates to trust. The default is to use Mozilla’s bundle, the same one used by Firefox.
- client_cert – A certificate to authenticate the client to the server
-
send_request
(method, path_components, body='', query_params=None)[source]¶ Does exactly the same as the method from the superclass, but also prettyprints the request and response if the
IEVV_ELASTICSEARCH_PRETTYPRINT_ALL_REQUESTS
setting isTrue
.
- urls – A URL or iterable of URLs of ES nodes. These can be full
URLs with port numbers, like
-
class
ievv_opensource.ievv_elasticsearch.search.
Connection
[source]¶ Bases:
ievv_opensource.utils.singleton.Singleton
Singleton wrapper around
pyelasticsearch.ElasticSearch
.We do not try to wrap everything, instead we use the pyelasticsearch API as it is, and add extra features that makes it easier to use with IEVV and Django. We provide shortcuts for the most commonly used methods of
pyelasticsearch.ElasticSearch
, some custom methods and we add some code to make unit testing easier.Usage:
from ievv_opensource.ievv_elasticsearch import search searchapi = search.Connection.get_instance() searchapi.bulk_index( index='contacts', doc_type='person', docs=[{'name': 'Joe Tester'}, {'name': 'Peter The Super Tester'}]) searchresult1 = searchapi.wrapped_search(query='name:joe OR name:freddy', index='contacts') searchresult2 = searchapi.wrapped_search(query={ 'query': { 'match': { 'name': { 'query': 'Joe' } } } }) print(searchresult1.total) for item in searchresult1: print(item.doc['name'])
-
elasticsearch
¶ The
pyelasticsearch.ElasticSearch
object.
-
clear_all_data
()[source]¶ Clear all data from ElasticSearch. Perfect for unit tests.
Only allowed when the
IEVV_ELASTICSEARCH_TESTMODE
-setting isTrue
.Usage:
class MyTest(TestCase): def setUp(self): self.searchapi = search.Connection.get_instance() self.searchapi.clear_all_data()
-
index
(*args, **kwargs)[source]¶ Wrapper around
pyelasticsearch.ElasticSearch.index()
.Works exactly like the wrapped function, except that we provide some extra features that makes testing easier. When the
IEVV_ELASTICSEARCH_TESTMODE
-setting isTrue
, we automatically runpyelasticsearch.ElasticSearch.refresh()
before returning.
-
bulk_index
(*args, **kwargs)[source]¶ Wrapper around
pyelasticsearch.ElasticSearch.bulk_index()
.Works exactly like the wrapped function, except that we provide some extra features that makes testing easier. When the
IEVV_ELASTICSEARCH_TESTMODE
-setting isTrue
, we automatically runpyelasticsearch.ElasticSearch.refresh()
before returning.
-
bulk
(*args, **kwargs)[source]¶ Wrapper around
pyelasticsearch.ElasticSearch.bulk()
.Works exactly like the wrapped function, except that we provide some extra features that makes testing easier. When the
IEVV_ELASTICSEARCH_TESTMODE
-setting isTrue
, we automatically runpyelasticsearch.ElasticSearch.refresh()
before returning.
-
search
(query, prettyprint_query=False, **kwargs)[source]¶ Wrapper around
pyelasticsearch.ElasticSearch.search()
.Works just like the wrapped function, except that
query
can also be anelasticsearch_dsl.Search
object, and you can only specify arguments as kwargs (no positional arguments).If
query
is anelasticsearch_dsl.Search
object, we convert it to a dict withquery.to_dict
before forwaring it to the underling pyelasticsearch API.Parameters: - query – A string, dict or
elasticsearch_dsl.Search
object. - prettyprint_query – If this is
True
, we prettyprint the query before executing it. Good for debugging.
- query – A string, dict or
-
refresh
(*args, **kwargs)[source]¶ Wrapper around
pyelasticsearch.ElasticSearch.refresh()
.Works exactly like the wrapped function.
-
delete_index
(*args, **kwargs)[source]¶ Wrapper around
pyelasticsearch.ElasticSearch.delete_index()
.Works exactly like the wrapped function.
-
delete
(*args, **kwargs)[source]¶ Wrapper around
pyelasticsearch.ElasticSearch.delete()
.Works exactly like the wrapped function.
-
get
(*args, **kwargs)[source]¶ Wrapper around
pyelasticsearch.ElasticSearch.get()
.Works exactly like the wrapped function.
-
wrapped_get
(*args, **kwargs)[source]¶ Just like
get()
, but we return aSearchResultItem
instead of the raw search response.
-
get_or_none
(*args, **kwargs)[source]¶ Works like
get()
, but instead of raising an exception, we returnNone
if the requested object does not exist.
-
wrapped_get_or_none
(*args, **kwargs)[source]¶ Works like
wrapped_get()
, but instead of raising an exception, we returnNone
if the requested object does not exist.
-
wrapped_search
(*args, **kwargs)[source]¶ Just like
search()
, but we return aSearchResultWrapper
instead of the raw search response.
-
paginated_search
(query, page_number=0, page_size=100, resultitemwrapper=None, **kwargs)[source]¶ Performs a search much like
wrapped_search()
, but limit the size of the result topage_size
, and start the results atpage_number * page_size
.Parameters: - page_number – The page number to retrieve.
- page_size – The size of each page.
- query – A query dict for
pyelasticsearch.ElasticSearch.search()
. We add thesize
andfrom
keys to this dict (calculated frompage_number
andpage_size
. - resultitemwrapper – Forwarded to
Paginator
. - kwargs – Forwarded to
wrapped_search()
alon withquery
.
Returns: The search results wrapped in a
Paginator
.Return type:
-
search_all
(**kwargs)[source]¶ Get all documents in the index. Nice for testing and debugging of small datasets. Useless in production.
**kwargs
are forwarded tosearch()
, but the query argument is added automatically.
-
wrapped_search_all
(**kwargs)[source]¶ Just like
search_all()
, but wraps the results in aSearchResultWrapper
.
-
-
class
ievv_opensource.ievv_elasticsearch.search.
SearchResultItem
(search_hit)[source]¶ Bases:
object
Wrapper around a single dictionary in the
hits.hits
list of the result returned bypyelasticsearch.ElasticSearch.search()
.-
id
¶ Returns the value of the
_id
key of the search hit.
-
index
¶ Returns the value of the
_index
key of the search hit.
-
score
¶ Returns the value of the
_score
key of the search hit.
-
source
¶ Returns the value of the
_source
key of the search hit.
-
doc_type
¶ Returns the value of the
_type
key of the search hit.
-
-
class
ievv_opensource.ievv_elasticsearch.search.
SearchResultWrapper
(searchresult)[source]¶ Bases:
object
An efficient wrapper around the data returned by
pyelasticsearch.ElasticSearch.search()
.Parameters: searchresult – Data returned by pyelasticsearch.ElasticSearch.search()
.-
total
¶ Returns the total number of hits.
-
retrieved_hits_count
¶ Returns the number of retrieved hits.
-
first
()[source]¶ Shortcut for getting the first search result as a
SearchResultItem
.
-
-
class
ievv_opensource.ievv_elasticsearch.search.
Paginator
(searchresultwrapper, page_number, page_size=100, resultitemwrapper=None)[source]¶ Bases:
object
Paginator for
ievv_opensource.ievv_elasticsearch.search.SearchResultWrapper
.The paginator counts the first page as 0, the second as 1, and so on.
Parameters: - searchresultwrapper – A
ievv_opensource.ievv_elasticsearch.search.SearchResultWrapper
. - page_number – The current page number.
- page_size – Number of items per page.
- resultitemwrapper – A class/callable that takes a single item in
the
searchresultwrapper
and does something with it before returning it when iterating the search result. Defaults to just returning the item as returned fromsearchresultwrapper.__iter__
.
-
total_items
¶ Returns the number of items in total in all pages.
-
number_of_items_in_current_page
¶ Returns the number of items in the current page.
-
page_has_content
(pagenumber)[source]¶ Check if the given
pagenumber
is within the total number of items in the givensearchresultwrapper
.Returns: A boolean.
-
current_page_has_content
()[source]¶ Check if current page is within the total number of items in the given
searchresultwrapper
.Returns: A boolean.
- searchresultwrapper – A
autoindex API¶
ievv_opensource.ievv_elasticsearch.autoindex.AbstractIndex |
Base class for describing a search index. |
ievv_opensource.ievv_elasticsearch.autoindex.AbstractDocument |
Base class for indexable documents for AbstractIndex . |
ievv_opensource.ievv_elasticsearch.autoindex.AbstractDictDocument |
Extends AbstractDocument to make it easy to put dicts in the database. |
ievv_opensource.ievv_elasticsearch.autoindex.Registry |
Registry of AbstractIndex objects. |
ievv_opensource.ievv_elasticsearch.autoindex.MockableRegistry |
A non-singleton version of Registry . |
This module defines a Registry
of objects that takes care of automatically
updating the search index when we detect changes to the data in a
data store. The data store can be anything you like (Django ORM,
MongoDB, …) - our examples use Django ORM.
This is completely decoupled from the ievv_opensource.ievv_elasticsearch.search
API.
-
class
ievv_opensource.ievv_elasticsearch.autoindex.
AbstractDocumentMeta
[source]¶ Bases:
type
Metaclass for
AbstractDocument
.
-
class
ievv_opensource.ievv_elasticsearch.autoindex.
AbstractDocument
[source]¶ Bases:
object
Base class for indexable documents for
AbstractIndex
.-
doc_type
= None¶ The document type to store this as in the index.
-
index_name
= None¶ The name of the index this document belongs to. This is set by
AbstractIndex
__init__ usingset_index_name_for_all_document_classes()
.
-
get_document
()[source]¶ Get document for the
doc
argument ofpyelasticsearch.ElasticSearch.index_op()
.
-
get_id
()[source]¶ Get the ID to use for the indexed document. Defaults to
None
, which means that a new document will be added to the index.
-
get_parent_id
()[source]¶ Get the parent-child mapping parent ID document to use for the indexed document. This should only be overridden if you have a parent specified
Defaults to
None
, which means that noparent
will be sent during indexing operations.
-
get_index_op_kwargs
()[source]¶ Get kwargs for
pyelasticsearch.ElasticSearch.index_op()
.You should not need to override this. Override
get_document()
,get_meta()
anddoc_type
.
-
classmethod
get_mapping_properties
()[source]¶ Get the mapping properties for custom mappings for this document type. You only need to specify those mappings you do not want elasticsearch to create automatically.
If you do not have any mappings, return
None
(or do not override).Examples
Simple example:
class MyDocument(autoindex.AbstractDocument): @classmethod def get_mapping_properties(cls): return { 'slug': { 'type': 'string', 'index': 'not_analyzed' }, 'author': { 'username': { 'type': 'string', 'index': 'not_analyzed' } } }
-
classmethod
get_mapping_parent_type
()[source]¶ Get the type of the parent document for parent-child mapping.
Lets say you have a Movie document, and want to create a parent-child relationship from the Category document with doc_type
category
to the Movie. In the Movie document class, you would have to:- Override this method and return
"category"
. get_parent_id()
and return the ID of the category.
- Override this method and return
-
-
class
ievv_opensource.ievv_elasticsearch.autoindex.
AbstractDictDocument
(document, id)[source]¶ Bases:
ievv_opensource.ievv_elasticsearch.autoindex.AbstractDocument
Extends
AbstractDocument
to make it easy to put dicts in the database.Parameters: - document – A dict that pyelasticsearch can convert to JSON.
- id – The ElasticSearch id of the document. Set to
None
to autocreate one.
-
class
ievv_opensource.ievv_elasticsearch.autoindex.
AbstractIndex
[source]¶ Bases:
object
Base class for describing a search index.
To register an index:
- Create a subclass of
AbstractIndex
and implementiterate_all_documents()
and overridedocument_classes
. - Register the index with
Registry
.
Examples
Minimal implementation for indexing a Django Product model:
from ievv_opensource.ievv_elasticsearch import searchindex class ProductDocument(searchindex.AbstractDictDocument): doc_type = 'product' class ProductIndex(searchindex.AbstractIndex): name = 'products' document_classes = [ ProductDocument ] def iterate_all_documents(self): for product in Product.objects.iterator(): yield ProductDocument({ 'name': product.name, 'price': product.price }, id=product.pk)
If you want a more general search index of sellable items, you could do something like this:
from ievv_opensource.ievv_elasticsearch import searchindex class ProductDocument(searchindex.AbstractDictDocument): doc_type = 'product' class ServiceDocument(searchindex.AbstractDictDocument): doc_type = 'service' class SellableItemIndex(searchindex.AbstractIndex): name = 'sellableitems' def iterate_all_documents(self): for product in Product.objects.iterator(): yield ProductDocument({ 'name': product.name, 'price': product.price, 'quantity': product.quantity }, id=product.pk) for service in Service.objects.iterator(): yield ServiceDocument({ 'name': service.name, 'price': service.price, }, id=service.pk)
You could also move the document creation into the index document classes like this:
class ProductDocument(searchindex.AbstractDictDocument): doc_type = 'product' def __init__(self, product): self.product = product def get_id(self): return self.product.id def get_document(self): return { 'name': self.product.name, 'price': self.product.price, 'quantity': self.product.quantity } class SellableItemIndex(searchindex.AbstractIndex): # ... same as above def iterate_all_documents(self): for product in Product.objects.iterator(): yield ProductDocument(product) # ...
-
name
= None¶ The name of the index. Must be set in subclasses.
-
bulk_index_docs_per_chunk
= 500¶ The number of docs to index per chunk when bulk updating the index.
-
bulk_index_bytes_per_chunk
= 10000¶ The number of bytes to index per chunk when bulk updating the index.
-
document_classes
= []¶ The
AbstractDocument
classes used in this index. Can also be overridden viaget_document_classes()
.
-
set_index_name_for_all_document_classes
()[source]¶ Called by __init__ to set the
AbstractDocument.index_name
of all documents indocument_classes
.
-
create
()[source]¶ Create the index and put any custom mappings.
You should not need to override this, instead you should override
get_document_classes()
(andAbstractDocument.get_mapping_properties()
), andget_settings()
.
-
get_settings
()[source]¶ Override this to provide settings for
pyelasticsearch.ElasticSearch.create_index()
(which is called bycreate()
.
-
get_document_classes
()[source]¶ Returns an iterable of the
AbstractDocument
classes used in this index. Defaults todocument_classes
.
-
get_document_classes_for_mapping
()[source]¶ Get the document classes for mapping. You normally do not have to override this - it only return
get_document_classes()
reversed. It is reversed because parent-child mappings have to be created in the child before the parent mapping can be created, but you normally want to index parents before children.
-
create_mappings
()[source]¶ Create mappings.
You should not need to override this, but instead you should override
get_document_classes()
(andAbstractDocument.get_mapping_properties()
).
-
iterate_all_documents
()[source]¶ Iterate over all documents returning documents that are ready to be added to the index.
Returns: An iterable of AbstractDocument
.
-
iterate_important_documents
()[source]¶ Just like
iterate_all_documents()
, but just yield the most important documents in case of a complete search index wipeout/rebuild.This is typically the newest and most important documents in the database.
Defaults to returning an empty list.
-
index_items
(index_documents)[source]¶ Index the given index_documents.
Iterates over the given
index_documents
, and send documents toievv_opensource.ievv_elasticsearch.search.Connection.bulk()
in batches ofIEVV_ELASTICSEARCH_INDEX_BATCH_SIZE
index_documents.Parameters: index_documents – An iterable of AbstractDocument
.
-
register_index_update_triggers
()[source]¶ Override this to register behaviors that trigger updates to the index. This is typically something like this:
- Register one or more post_save signals that updates the index in realtime (be very careful with this since it can easily become a bottleneck).
- Register one or more post_save signals that updates the index via a Celery job or some other background queue.
Does nothing by default, so it is up to you to override it if you want to register any triggers.
-
classmethod
get_instance
()[source]¶ Get an instance of this class.
Use this instead of instanciating the class directly.
-
rebuild_index
()[source]¶ Rebuild this index completely.
Very useful when writing tests, but probably a bit less than optimal in production code/batch tasks unless you have a really small index. In production you should most likely want to create a management command to rebuild the index with the most recent/most important documents beeing indexed first.
- Create a subclass of
-
class
ievv_opensource.ievv_elasticsearch.autoindex.
Registry
[source]¶ Bases:
ievv_opensource.utils.singleton.Singleton
Registry of
AbstractIndex
objects.Examples
First, define an index (see
AbstractIndex
).Register the searchindex with the searchindex registry via an AppConfig for your Django app:
from django.apps import AppConfig from ievv_opensource.ievv_elasticsearch import searchindex from myapp import elasticsearch_indexes class MyAppConfig(AppConfig): name = 'myapp' def ready(self): searchindex.Registry.get_instance().add(elasticsearch_indexes.SellableItemIndex)
-
get
(indexname)[source]¶ Get the index named
indexname
.- Returns: An
AbstractIndex
orNone
if no index matching - the given
indexname
is found.
- Returns: An
-
-
class
ievv_opensource.ievv_elasticsearch.autoindex.
MockableRegistry
[source]¶ Bases:
ievv_opensource.ievv_elasticsearch.autoindex.Registry
A non-singleton version of
Registry
. For tests.Typical usage in a test:
class MockSearchIndex(searchindex.AbstractIndex): name = 'myindex' # ... mockregistry = searchindex.MockableRegistry() mockregistry.add(searchindex.MockSearchIndex()) with mock.patch('ievv_opensource.ievv_elasticsearch.searchindex.Registry.get_instance', lambda: mockregistry): pass # ... your code here ...
jsondecode API¶
Utilities for decoding the JSON returned by ElasticSearch.
viewhelpers API¶
ievv_opensource.ievv_elasticsearch.viewhelpers.searchview.ViewMixin |
Makes it a bit easier to search with paging. |
ievv_opensource.ievv_elasticsearch.viewhelpers.searchview.View |
A Django TemplateView with ViewMixin . |
ievv_opensource.ievv_elasticsearch.viewhelpers.searchview.SortMixin |
Mixin class for sort-keyword in the querystring based sort (E.g.: ?o=name ). |
ievv_opensource.ievv_elasticsearch.viewhelpers.searchview.SearchMixin |
Mixin class that makes it slightly easier to add search via a querystring attribute (defaults to ?s=<search_string> ). |
-
class
ievv_opensource.ievv_elasticsearch.viewhelpers.searchview.
ViewMixin
[source]¶ Bases:
object
Makes it a bit easier to search with paging.
Examples
Minimal example:
class MyView(TemplateView, searchview.ViewMixin): template_name = 'myapp/mytemplate.html' def get_search_query(self): return { 'match_all': {} } def get_context_data(self, **kwargs): context = super().get_context_data(**kwargs) context['searchpaginator'] = self.get_paginator()
A more full featured example:
class MyView(TemplateView, searchview.ViewMixin): template_name = 'myapp/mytemplate.html' page_size = 30 paging_querystring_attribute = 'page' def get_search_query(self): return { 'match_all': {} } def get_search_sort(self): return {'name': {'order': 'asc'}} def get_context_data(self, **kwargs): context = super().get_context_data(**kwargs) context['searchpaginator'] = self.get_paginator()
You should normally just need to override:
get_search_query()
.get_search_sort()
(optional).get_page_size()
orpage_size
(optional).get_resultitemwrapper()
(optional).get_paging_querystring_attribute()
orpaging_querystring_attribute
(optional).
And call
get_paginator()
to retrieve results that you can iterate and ask for pagination information.-
page_size
= 100¶ The number of items per page. Defaults to
100
. Seeget_page_size()
.
-
search_index
= None¶ See
get_search_index()
. Defaults toNone
.
-
search_doc_type
= None¶ See
get_search_doc_type()
. Defaults toNone
.
-
paging_querystring_attribute
= 'p'¶ The querystring attribute to use for paging. Defaults to
p
. Used byget_paging_querystring_attribute()
.
-
get_paging_querystring_attribute
()[source]¶ The querystring attribute to use for paging. Defaults to
paging_querystring_attribute
.
-
get_current_page_number
()[source]¶ Get the current page number from
request.GET[self.get_paging_querystring_attribute()]
.You can override this if you want to get the page number some other way.
-
get_search_index
()[source]¶ Get the search index name.
Defaults to
search_index
.
-
get_search_doc_type
()[source]¶ Get the document types to search. Can be a single document type or an iterable of document types.
Defaults to
search_doc_type
.
-
get_search_query
()[source]¶ Get the query attribute of the elasticsearch query.
You MUST override this.
While
get_search_full_query()
returns something like:{ 'query': { 'match_all': {} }, 'size': 20, 'from': 40 }
This method should only return the value of the
query
key:{ 'match_all': {} }
-
get_search_sort
()[source]¶ Get the sort dict for the search query.
While
get_search_full_query()
returns something like:{ 'query': { 'match_all': {} }, 'sort': {'name': {'order': 'asc'}}, 'size': 20, 'from': 40 }
This method should only return the value of the
sort
key:{'name': {'order': 'asc'}}
Defaults to
None
, which means no sorting is performed.
-
get_search_full_query
()[source]¶ Builds the full ElasticSearch query dict including paging.
You should normally not override this directly. Override
get_search_query()
andget_search_sort()
instead.
-
get_paginated_search_kwargs
()[source]¶ Get the kwargs for
ievv_opensource.ievv_elasticsearch.search.Connection#paginated_search()
.
-
get_resultitemwrapper
()[source]¶ See the
resultitemwrapper
argument forievv_opensource.ievv_elasticsearch.search.Paginator
.
-
get_paginator
()[source]¶ Performs the search and wraps it in a
ievv_opensource.ievv_elasticsearch.search.Paginator
.Raises: django.http.response.Http404
if the search does not match- any items. You will typically catch this exception and
- show a message in a normal search setting.
-
class
ievv_opensource.ievv_elasticsearch.viewhelpers.searchview.
View
(**kwargs)[source]¶ Bases:
django.views.generic.base.TemplateView
,ievv_opensource.ievv_elasticsearch.viewhelpers.searchview.ViewMixin
A Django TemplateView with
ViewMixin
.For usage example, see
ViewMixin
(just inherit from this class instead of TemplateView and ViewMixin)Constructor. Called in the URLconf; can contain helpful extra keyword arguments, and other things.
-
class
ievv_opensource.ievv_elasticsearch.viewhelpers.searchview.
SortMixin
[source]¶ Bases:
object
Mixin class for sort-keyword in the querystring based sort (E.g.:
?o=name
).This MUST be mixed in before
View
(orViewMixin
), since it overridesViewMixin.get_search_sort()
.Examples
Simple example of a map where
?o=name
sorts by name ascending and?o=created
sorts by created datetime descending:class MySortView(searchview.SortMixin, searchview.View): default_sort_keyword = 'name' sort_map = { 'name': {'name': {'order': 'asc'}}, 'created': {'created_datetime': {'order': 'desc'}}, } def get_search_query(self): return { 'match_all': {} } def get_context_data(self, **kwargs): context = super().get_context_data(**kwargs) context['searchpaginator'] = self.get_paginator()
The default if
?o
is not specified will be to sort by name ascending.You should normally just need to override:
If you do not get the sort keyword from the querystring, you also need to override
get_sort_keyword()
.If you do not want to use
o
as the querystring attribute for sort keywords, you need to overrideget_sort_querystring_attribute()
orsort_querystring_attribute
.-
sort_querystring_attribute
= 'o'¶ The querystring attribute to use for sort. Used by
get_sort_querystring_attribute()
. Defaults too
(for ordering). We useo
instead ofs
to avoid collision withSearchMixin
.
-
default_sort_keyword
= 'default'¶ The default sort keyword to use when ordering is not specified in the querystring. Defaults to
"default"
.
-
sort_map
= {}¶ See
get_sort_map()
.
-
get_sort_querystring_attribute
()[source]¶ The querystring attribute to use for sort. Defaults to
sort_querystring_attribute
.
-
get_default_sort_keyword
()[source]¶ Get the default sort keyword to use when ordering is not specified in the querystring. Defaults to
default_sort_keyword
.
-
get_sort_keyword
()[source]¶ Get the sort keyword. Defaults to getting it from the querystring argument defined by
get_sort_querystring_attribute()
, and falling back toget_default_sort_keyword()
.
-
get_sort_map
()[source]¶ Get a mapping object that maps keywords to sort dicts compatible with elasticsearch. Defaults to
sort_map
.
-
get_search_sort_by_keyword
(keyword)[source]¶ Get the elasticsearch sort dict by looking up the given
keyword
inget_sort_map()
.If the given
keyword
is not inget_sort_map()
, we fall back onget_default_sort_keyword()
.
-
get_search_sort
()[source]¶ Overrides
ViewMixin.get_search_sort()
and gets the value of the sort dict by viaget_search_sort_by_keyword()
withget_sort_keyword()
as the keyword argument.This means that you should not override this method, but instead override:
-
-
class
ievv_opensource.ievv_elasticsearch.viewhelpers.searchview.
SearchMixin
[source]¶ Bases:
object
Mixin class that makes it slightly easier to add search via a querystring attribute (defaults to
?s=<search_string>
).This is perfect for views that use ElasticSearch for traditional search where a user types in a string, and we wish to search some fields for that string.
This MUST be mixed in before
View
(orViewMixin
) , since it overridesViewMixin.get_search_query()
.Can safely be used with
SortMixin
. The order ofSortMixin
andSearchMixin
does not matter, but they must both be mixed in beforeView
(orViewMixin
).Examples
Minimal example:
class MySearchView(searchview.SearchMixin, searchview.View): search_query_fields = ['name', 'email'] def get_context_data(self, **kwargs): context = super().get_context_data(**kwargs) context['searchpaginator'] = self.get_paginator()
-
search_querystring_attribute
= 's'¶ The querystring attribute to use for sort. Used by
get_sort_querystring_attribute()
. Defaults tos
.
-
search_query_fields
= []¶ List of fields for
get_search_query_fields()
. Defaults to empty list, so you need to set this, or overrideget_search_query_fields()
.
-
get_search_querystring_attribute
()[source]¶ The querystring attribute to use for search. Defaults to
search_querystring_attribute
.
-
clean_search_string
(search_string)[source]¶ Can be overridden to clean/modify the search_string.
Does nothing by default.
Used by
get_search_string()
.
-
get_search_string
()[source]¶ Get the search string.
We get the search string from the querystring, and clean it with
clean_search_string()
before returning it.
-
get_search_query_fields
()[source]¶ Get the fields for the
multi_match
query performed byget_search_query_with_search_string()
.Defaults to
search_query_fields
-
get_search_query_with_search_string
(search_string)[source]¶ Called by
get_search_query()
whenget_search_string()
returns something.Defaults to a
multi_matach
query with the fields returned byget_search_query_fields()
.You will not need to override this for simple cases, but for more complex queries with boosting, filtering, etc. you will most likely have to override this.
-
get_search_query_without_search_string
()[source]¶ Called by
get_search_query()
whenget_search_string()
returns empty string orNone
.Defaults to a
match_all
query.
-
get_search_query
()[source]¶ Overrides
ViewMixin.get_search_query()
and splits the logic into two separate states:- We have a search string, call
get_search_query_with_search_string()
. - We do not have a search string, call
get_search_query_without_search_string()
.
You should not need to override this method, but instead override:
get_search_fields()
or perhapsget_search_query_with_search_string()
(for advanced cases).- Perhaps
get_search_query_without_search_string()
.
- We have a search string, call
-