File Search and Query

Organize, sort, and filter to find the data you need, when you need it

The larger your data storage grows, the harder it becomes to search and find the files that you want. Without proper file organization, classification, and indexing, users spend a significant amount of their productive time foraging for the data they need.

DataCore Swarm object storage platform addresses and solves this problem in the most efficient and user-friendly way. Swarm is seamlessly integrated with the Elasticsearch search engine, giving you a wide range of exabyte-scale search and analytics options. Queries and searches can be conducted by command line through the Swarm web-based Content Portal or via the Swarm API.

Swarm allows users to add any number of custom metadata attributes to files stored as objects allowing greater flexibility in performing queries. Search, query, and retrieve operations happen at near-real time speed, even when searching across millions of files occupying exabytes of storage.

How Elasticsearch Enables Rapid Search Operations

Elasticsearch is a distributed, RESTful open-source search and analytics engine built for the cloud that indexes object metadata extremely fast and lets users perform ad hoc searches on the attributes and metadata of their stored objects. Integration of Elasticsearch with Swarm delivers a rich metadata-based search experience allowing users to find the files they need in no time.

  • Search queries issued by a user/application reach the Swarm Cluster.
  • The Swarm Cluster automatically connects to the Elasticsearch service through a Search Feed and transmits the search queries and metadata.
  • Elasticsearch servers index the metadata and service the search queries.
  • The Search Feed carries back the replies from Elasticsearch to the Swarm Cluster, which the user/application is able see on the Swarm Content Portal.

Minutes of manual file search operations are reduced to just a few seconds, saving manual effort, and improving user productivity.

elasticsearch enables rapid search operations

Highlights of Swarm’s Search Capabilities

The Swarm Search capabilities map 1:1 with S3 metadata and offer many benefits including:

  • Actionable insights with targeted analysis
  • Dynamic organization of content using classification, keywords, and descriptive content, with multiple ways to track that content
  • Integrated search stack optimized within the storage system

Web-Based Portal

Quickly define, execute, and view results from any popular web-based browser.

Queries from Swarm CLI

Execute any supported query from the Swarm command line interface and gain direct access to content.

Robust HTTP RESTful Interface

All search functionality is also available via the Swarm SCSP API for integration into third-party applications.

Third-Party App Integrations

Plugs directly into Amazon S3 allowing access to data from Swarm.

Custom Metadata

Add custom metadata to any file, at any time via a web-browser or the API.

Filter & Organize Large Result Sets

Define exactly how data is organized, previewed, and filtered to limit results or narrow searches.

Query Saving for Rapid Reuse

All searches are dynamic and may be saved for rapid re-use to later speed up query building.

Export Results to JSON or XML

Search results can be exported to JSON or XML for integration into analytics platforms or processes.

Leverages the ELK Stack

Enabled by Elasticsearch and seamlessly works with Logstash and Kibana for analysis and visualization.

Get Started with Swarm, Software-Defined Object Storage