The larger your data storage grows, the harder it becomes to search and find the files that you want. Without proper file organization, classification, and indexing, users spend a significant amount of their productive time foraging for the data they need.
DataCore Swarm object storage platform addresses and solves this problem in the most efficient and user-friendly way. Swarm is seamlessly integrated with the Elasticsearch search engine, giving you a wide range of exabyte-scale search and analytics options. Queries and searches can be conducted by command line through the Swarm web-based Content Portal or via the Swarm API.
Swarm allows users to add any number of custom metadata attributes to files stored as objects allowing greater flexibility in performing queries. Search, query, and retrieve operations happen at near-real time speed, even when searching across millions of files occupying exabytes of storage.
How Elasticsearch Enables Rapid Search Operations
Elasticsearch is a distributed, RESTful open-source search and analytics engine built for the cloud that indexes object metadata extremely fast and lets users perform ad hoc searches on the attributes and metadata of their stored objects. Integration of Elasticsearch with Swarm delivers a rich metadata-based search experience allowing users to find the files they need in no time.
- Search queries issued by a user/application reach the Swarm Cluster.
- The Swarm Cluster automatically connects to the Elasticsearch service through a Search Feed and transmits the search queries and metadata.
- Elasticsearch servers index the metadata and service the search queries.
- The Search Feed carries back the replies from Elasticsearch to the Swarm Cluster, which the user/application is able see on the Swarm Content Portal.
Minutes of manual file search operations are reduced to just a few seconds, saving manual effort, and improving user productivity.
Highlights of Swarm’s Search Capabilities
The Swarm Search capabilities map 1:1 with S3 metadata and offer many benefits including:
- Actionable insights with targeted analysis
- Dynamic organization of content using classification, keywords, and descriptive content, with multiple ways to track that content
- Integrated search stack optimized within the storage system
Web-Based Portal
Quickly define, execute, and view results from any popular web-based browser.
Queries from Swarm CLI
Execute any supported query from the Swarm command line interface and gain direct access to content.
Robust HTTP RESTful Interface
All search functionality is also available via the Swarm SCSP API for integration into third-party applications.
Third-Party App Integrations
Plugs directly into Amazon S3 allowing access to data from Swarm.
Custom Metadata
Add custom metadata to any file, at any time via a web-browser or the API.
Filter & Organize Large Result Sets
Define exactly how data is organized, previewed, and filtered to limit results or narrow searches.
Query Saving for Rapid Reuse
All searches are dynamic and may be saved for rapid re-use to later speed up query building.
Export Results to JSON or XML
Search results can be exported to JSON or XML for integration into analytics platforms or processes.
Leverages the ELK Stack
Enabled by Elasticsearch and seamlessly works with Logstash and Kibana for analysis and visualization.
Get Started with Swarm, Software-Defined Object Storage
Store your growing data on scalable & secure active archive. No more insane cloud costs!
Join thousands of IT pros who have benefitted from DataCore solutions.
“Scalability is one of the slickest things about Swarm. One sweet thing with the platform is when you drop a new node in, it builds itself from the ground up. If you want to add a 72 TB or PB node you just drop it in.”
John Gillam – British Telecommunications
CTO for Cloud Services