Posts

Showing posts with the label solr

Connect Django Haystack to Solr Cloud

At BV FAPESP (www.bv.fapesp.br) we use Solr as the searchengine backend, and a library called Haystack to tie Solr to Django. In 2018, me and my team wrote a Python/Django library to use with Apache Solr in cloud mode. We were avoiding the use of Django/Haystack library, since there were some features not supported, like grouping, Streaming Expressions, Graph Analysis. So far so good, before the end of the project I had in production environment Solr Cloud running smoothly, but I still had a single Solr running with Haystack, because we didn't re-code the whole system, and there still exist a legacy using Haystack. To turn-off the single Solr, we moved all documents to Solr Cloud and connected Haystack to it. This is what I documented here, for myself and maybe you, trying to make the same. Step-by-step There is Solr Cloud python backend for Haystack, that you can find here: https://github.com/django-haystack/django-haystack/pull/1580/commits/13df4a9e69ececd5567636085df4...

Solr - Graph traversal query

Image
In this document I will show you how I am using some distinct technology layers to display graph data from BV FAPESP, stored inside Solr. FAPESP´s virtual library (BV FAPESP) is the information system, source of the referential data, of funded projects from São Paulo funding agency. These projects holds relation to each other in a three structure, a specialized form of a graph structure. BV FAPESP data projects structure In the relational database, the projects are linked by foreign keys, the most simple form to store the three structure relationship. But the main data source of BV FAPESP, is not a relational database, but the search engine Solr. Solr is a search engine, that uses the inverted index structure which is very fast at query time. The initial approach At the early stages of BV FAPESP, the associated projects were indexed with some kind of information from their parent, to allow the inverted search to get all children of a subset of projects. Everything is...

Solr Facet add link

Image
This is about how to link the resulting facet data from Solr on your web application. I am using Django as the front-end framework. But this solution applies to any language. The problem You have modeled your data and indexed to Solr. You will show facets on your web application to facilitate the navigation on your site to the users. You get faceted data from Solr; so far so good. The problem arises when you need to link your facet item to any destination other than the value of the facet itself, because Solr faceted data returns only the field value and count of occurences. The example below is a simplification of the Solr facet web page 1 , and show you how the json facet returns: {   "facet_counts" :{     "facet_queries" :{},     "facet_fields" :{       "categories" :[         "electronics" , 14 ,         "currency" , 4 , ...