How to build an effective search portal for advanced data?
We are floating in a sea of data. There is so much data that finding what you are looking for may be extremely difficult, especially if we’re looking for advanced data and products derived from it. Whether it’s remote sensing, or any other kind of earth observation data, without a dedicated solution that gathers the thematic data in one place, it may be nearly impossible.
Many organizations solve this issue by building an advanced search portal with managed data uptake. This is one way to go, but we are often seeing that they aren’t built in a sustainable fashion and don’t fully serve their mission. In this article, we explain how to build an effective search engine for advanced data. We will take, taking our recent realization for the Group on Earth Observations of the European Space Agency – GEOSS portal, a project we’ve recently implemented for Group on Earth Observations to enable remote sensing and earth observation data search portal, as our example.
Advanced portal objectives
A search portal essentially allows you to discover data connects data users with data owners, so for your search engine to work effectively, it has to “know” where the data a user is looking for is located and understand the search queries when they are entered by the user.
For instance, GEOSS portal allows users to discover and access Earth observation data and resources from different providers from all over the world. It’s an online discovery and access point to the ever-growing quantities of remote sensing data from satellites, airplanes, drones and in-situ sensors at global, regional and local scales. It facilitates data and information accessibility and interoperability in the Earth Observation field.
Objective 1: Enabling data discovery
To build an advanced search portal that is able to discover relevant data within a specific field, a number of conditions must be met. Take a look at the below:
- Define the dictionary
Begin by developing the relevant ontology for semantic searching of products and services for the relevant sector. This isn’t an easy task, but it’s extremely important for the efficiency of your search engine. You need to be able to predict what the users will be looking for and how will they define it as a search query and build a relevant semantic tree accordingly index relevant terms accordingly. It’s also important to do this work thoroughly as it will have a great impact on how the search portal “understands” the search queries being entered.
- Configure data access
Your search portal must know what data to look for and where to find it. GEOSS portal was built with GEO Discovery and Access Broker, which is the primary mechanism for discovering and accessing all data. It’s built o APIs, which allows data providers to share resources without having to make major changes to their technology or standards. A good search portal will integrate with external search engines to ensure more precise search and discovery.
- Enable contextualization
An effective search engine will also be able to gather and analyze the metadata from public data sources to better contextualize search queries. This will help increase user understanding and engagement. GEOSS portal features advanced metadata analytics and prioritized indexation that enable a more precise search. The portal is also able to improve metadata to increase the discoverability of public data sets.
- Monitor traffic performance
This feature will allow to improve the user interface. Traffic monitoring will help you understand e.g. how many steps a user had to make before they took a specific action, e.g. downloaded the document. It allows simplifying user journey in the long run. (…)
Objective 2: Enabling user engagement
Configure data preview
At this stage, you have a data-gathering system in place, but since the data is coming from different sources, the quality and format will be different. Oftentimes it will gather raw data, which will be meaningless to an average user. To make your search portal effective, it must be able to create supporting content that supplements this raw data, i.e. visualizations through interactive tables, diagrams, maps, images and other relevant tools.
Enable advanced search features
The user must be able to narrow down the search, whether it’s by location, time, volume, or any other criteria. Your advanced search features will be closely related to the approach you choose for indexation, as defined in the ontology for the search engine. Allow users to filter through data as well.
Managing public data with custom-built software
GEOSS portal was built to ensure easy access to Earth Observation data. When the European Space Agency approached us to help them with this project, we discovered that the portal needed significant technological updates, visual redesign, as well as UX and usability enhancements to improve the search functionalities.
GEOSS decision-makers were looking for a user-friendly solution that would expand its audience. We navigated the project from planning through execution and implementation. As part of this work, we took over the hosting and deployment role to enhance project operations, prepared for the evolution of the portal with the elements listed above and then completely redesigned it to solve visual and UX issues.
As a result, GEOSS portal is now more stable, with modular architecture based on Liferay Portal, which can be easily fine-tuned as required. It has improved integration capacity thanks to the standardization of the API integration.
Thanks to enhanced collaboration, the overall quality of the results returned by the system improved. The users are now able to share the queries which return good quality results and highlight the high-quality datasets. The portal also measures user satisfaction as a way to help improve its KPIs.
If you’re interested in enhancing your data management portal, contact us directly and we’ll be happy to help.