Orchestrate Eases Complex Database Tasks

Doron Katz,  Consultant | Analyst | Journalist | Socialite
May. 28 2014, 11:00AM EDT

So, you are an indie developer, or a small to midsize business, without the resources to curate your own complex backend? Enter Orchestrate (formerly known as Orchestrate.io), a database-as-a-service (DBaaS) provider that manages a complex infrastructure of scalable storage and querying engines (NoSQL), each optimized to a particular task or query.

Orchestrate’s solutions are hosted on Amazon Web Services, across three zones in the U.S market. Company officials say plans are in place to open a European data center, allowing for further customer data replication coverage.

If you are a startup or an independent developer looking to release your minimum viable product (MVP), or a small to midsize business looking for a robust and scalable data infrastructure, Orchestrate will be a good fit. With a sophisticated algorithm capable of anticipating demand growths and spikes, Orchestrate’s estimations allow for complete uptime.

When you think about whether a solution fits your business model, you have to assess where your core competencies and priorities lie. If you don’t want to commit resources and time to managing (or “baby-sitting,” as Orchestrate puts it) your database infrastructure—but don’t want to compromise on scalability--this is an ideal service.

Orchestrate’s REST API and dashboard are extremely straightforward. Within 15 minutes of reading the walkthrough guides, you will be up and running with the product.

Orchestrate’s pricing model is quite competitive. And, with the ability to export your data, you have the flexibility to move to a more robust system should your company needs change.

Orchestrate isn’t for everyone. For example, if you have the capacity to manage your own AWS database and are looking for finer control over the load balancing of your databases, Orchestrate isn’t the best choice. It’s important to balance the costs of curating your own database back end and having valet-like management of your back-end infrastructure.

You also have to consider whether you require a relational database system (such as MySQL or SQL Server) or a NoSQL non-relational document-based database system. The choice, of course, depends on whether your business/application is more suited for hierarchical data storage, and whether you require vertical or horizontal scalability.

Putting Orchestrate to the Test

The Orchestrate service takes care of most of the tedious database administrative structuring and balancing tasks, allowing users to focus on core competencies. And, as a truly multitenant service, Orchestrate grows with your business as demands grow. This is why prominent enterprise customers such as Rovio--the company behind the popular Angry Birds mobile game--are relying on Orchestrate as part of their cloud infrastructure.

Right from the get-go, we were presented with a simple step-by-step walkthrough to help get started on object-model definitions. Orchestrate then provided a tailored set of code snippets that guided us through the process of accessing data via its REST API.

Orchestrate provides a concise dashboard that feeds users vital stats on how the back end is performing. During the coming months, Orchestrate officials say, the company will focus on building up its developer community base, in terms of resources, documentation, support and tutorials.

Full-Text Querying

As a NoSQL service, Orchestrate provides out-of-the-box full-text indexing through the Apache Lucerne Query Parser structure. This relieves developers of the responsibilities of having to set up a dedicated database just to sync and index their primary database. Developers can instead leverage the best of the SQL-less benefits, such as automatic indexing of every key/value pairing.

Graphing and Relationships

Another important facet of database modeling is working with object relationships, or graphing. This is somewhat tricky, especially with NoSQL database architectures like MongoDB. Orchestrate makes setting relationships between objects scalable without limiting the number of relationships users can construct. This allows for full traversing of collection/key relationships.

Events and Triggers

In Orchestrate lingo, Events are time-based objects, and the company provides a highly robust queued API mechanism that eliminates bottlenecks and database-locking. You simply set an event along with a type (such as update or tweet), associate it with an object and collection (table), and timestamp it in your API call. Orchestrate takes care of the rest.

GeoSpatial Indexing

The ability to power your objects with geospatial metadata is something Orchestrate is promising soon. The feature will allow developers to query and index objects contextually based on geographical locations.
 
In the coming months, company officials say, Orchestrate will introduce features including Batch Operations and Dashboard Cluster.

API Walkthrough

The best way to demonstrate how Orchestrate works is through an end-to-end walkthrough.

When you sign up for the free Orchestrate plan, you are presented with a dashboard that will let you create a new application. In the application page, you can set up your API keys to create collections (synonymous with database tables).

Using the provided online dashboard, we will walk through the process of setting up a fictitious Students Collection and some students.
 

After creating a collection, we were presented with a set of sample customized code snippets for testing API endpoints, without the need for a client (through Curl or an HTTP tool of your choice).
 

 

 {

   "FirstName": "Jason",

   "LastName": "Newell",

   "Grade": 11,

   "Classes": "Computer Studies, Earth Sciences, English, Calculus",

   "Bio": "A very creative student with good intution"

}

 

 
For the key-part, we specified a unique identifier: “Jason_Newell”.
Once we entered the details and clicked Play, we got the following HTTP response:
 

HTTP/1.1 201 Created

 Content-Type: application/json

 Date: Sun, 20 Apr 2014 04:43:03 GMT

 Location: /v0/Students/Jason_Newell/refs/e3c773a12ea8782d

 Server: nginx

 X-ORCHESTRATE-REQ-ID: 426f6770-c846-11e3-96ab-12313d2f9238

 Content-Length: 0

 Connection: keep-alive

We then created another student. (Rinse and repeat.)

{

    "FirstName": "Kate",

    "LastName": "Lewis",

    "Grade":10,

    "Classes": "English, Sociology, Geography, Calculus",

    "Bio": "A very creative student with good intution"                

}

 

It’s as simple as that: The process for HTTP PUT and the technique used is implemented pretty much the way in which you would do it using Curl or any other HTTP tool/method.
 

 {
    "count": 2,
    "results": [
        {
            "path": {
                "collection": "Students",
                "key": "Jason_Newell",
                "ref": "e3c773a12ea8782d"
            },
            "value": {
                "Bio": "A very creative student with good intution",
                "Classes": "Computer Studies, Earth Sciences, English, Calculus",
                "FirstName": "Jason",
                "Grade": 11,
                "LastName": "Newell"
            }
        },
        {
            "path": {
                "collection": "Students",
                "key": "Kate_Lewis",
                "ref": "2032feb771804d16"
            },
            "value": {
                "Bio": "A very creative student with good intution",
                "Classes": "English, Sociology, Geography, Calculus",
                "FirstName": "Kate",
                "Grade": 10,
                "LastName": "Lewis"
            }
        }
    ]
}

 

Using the GET option, we could either get a list of all the students or a particular student. Without entering any parameters, we would get the entire collection:
 

{
    "count": 2,
    "results": [
        {
            "path": {
                "collection": "Students",
                "key": "Jason_Newell",
                "ref": "e3c773a12ea8782d"
            },
            "value": {
                "Bio": "A very creative student with good intution",
                "Classes": "Computer Studies, Earth Sciences, English, Calculus",
                "FirstName": "Jason",
                "Grade": 11,
                "LastName": "Newell"
            }
        },
        {
            "path": {
                "collection": "Students",
                "key": "Kate_Lewis",
                "ref": "2032feb771804d16"
            },
            "value": {
                "Bio": "A very creative student with good intution",
                "Classes": "English, Sociology, Geography, Calculus",
                "FirstName": "Kate",
                "Grade": 10,
                "LastName": "Lewis"
            }
        }
    ]
}

 

We could also filter by a specific key.

We then decided to dig a bit deeper into the search capabilities of the platform. By selecting the search tab, we could test some query parameters.

By entering * in the search query, we got a result similar to the one above, using the wildcard that is inclusive of everything in the collection. To drill down a little further, we entered the query:
 

Looking at the students we created, only one has Sociology as a class, so the filter will produce only one student result:
 

{
    "count": 1,
    "results": [
        {
            "path": {
                "collection": "Students",
                "key": "Kate_Lewis",
                "ref": "2032feb771804d16"
            },
            "score": 0.1534264087677002,
            "value": {
                "Bio": "A very creative student with good intution",
                "Classes": "English, Sociology, Geography, Calculus",
                "FirstName": "Kate",
                "Grade": "11",
                "LastName": "Lewis"
            }
        }
    ],
    "total_count": 1
}

 

We could also include ranges in our queries, such as the following: Classes:Calculus AND Grade: [9 TO 12}.

There are various other powerful Lucee querying techniques you can apply to filter your data set according to your requirements, and they are quite straightforward.

During tests, we added and queried events on our project. We added the following JSON data and then selected the Play button:
 

{
    "Teacher": "Frank_Kosta",
    "Text": "Jason has a good focus and understanding of Calculus."
}

 

We continued to add JSON data, retaining the same properties for type and key. When we wanted to retrieve all the comments for a particular student, we went to the GET sub-tab of the Events tab. We got the following:
 

{
    "count": 2,
    "results": [
        {
            "ordinal": 6,
            "timestamp": 1397972748247,
            "value": {
                "Teacher": "Rachel_Wiess",
                "Text": "Jason is always the first student to put his hand up. Quite passionate about his class"
            }
        },
        {
            "ordinal": 5,
            "timestamp": 1397972709853,
            "value": {
                "Teacher": "Frank_Kosta",
                "Text": "Jason has a good focus and understanding of Calculus."
            }
        }
    ]
}

The final step of our mini-project walkthrough explored the use of graphs and the creation of relationships. To create a relationship between Students and Teacher, we created a Teacher JSON dataset in the Key/Value tab:
 

{

   "FirstName": "Rachel",

   "LastName": "Wiess",

   "Job": "Teacher",

   "Classes": "Computer Studies"

}

 

To create the relationship graph, we went back in the Graph tab.
 

We created another relationship, the traverse (or, another other way of thinking about something). To test our work, we went to the GET subtab of the Graph tab. Orchestrate let us define relationships within the same collection or across collections with ease, but without a lot of the constraints we are used to when working with relational databases.
 

 

{
    "count": 1,
    "results": [
        {
            "path": {
                "collection": "Students",
                "key": "Rachel_Wiess",
                "ref": "a407ec53349ba6b2"
            },
            "value": {
                "Classes": "Computer Studies",
                "FirstName": "Rachel",
                "Job": "Teacher",
                "LastName": "Wiess"
            }
        }
    ]
} 

API Summary

The Orchestrate Documentation can be found at orchestrate.io/docs. There, you will be presented with clear and concise information. Among other things, you will get information on how authentication works within the Orchestrate ecosystem, and along all the API calls/methods. Orchestrate also provides sample snippets in Curl and Java.
 

 

Pricing

Orchestrate prices its service according to what it terms MegaOp or MOp, which equates to either 1 million operations or 1 million API calls. Interestingly, Orchestrate does not charge for storage space but rather for API access.

Orchestrate provides three pricing tiers: The Starter plan costs $39 per month, allowing up to 10 MOps of usage; the Developer plan costs $149 per month, allowing 50 MOps of usage; and the Enterprise tier costs $749 per month with 100 Mops of usage.

All tiers include all administrative operations, such as monitoring and automated backing up. Database replication is handled by Orchestrate, as well.

Community and Support

No matter which tier you choose, you will have access to Orchestrate’s engaging community chatroom. From the Development tier and up, users get more personalized support, including private-chat and phone support. The Enterprise tier provides 24/7 support, while the Development tier provides support during business hours.

Doron Katz Focused on Lean Startup (Lean Analytics | Lean Customer Development), I embrace a vision of scalable lean code, manage projects with succinct discipline, with continuous customer development and hypotheses testing. Besides Project and Product Management, I enjoy coding in Objective-C and Swift. I am also a regular contributor on ProgrammableWeb, one of the most prominent API review websites, and I also regularly review technical books for O'Reilly Publishing.

Comments