livedomain Discussions Rss Feed

New Post: My questions

rofr — Sun, 11 Nov 2012 23:47:17 GMT

A single standalone LiveDB Server is of course a single point of failure. Failover clustering uses multiple nodes but is modeled after sql server database mirroring. Only the master accepts commands which are replicated to the slaves, either sync (safe mode) or async (high performance mode). Just guessing here, but it sounds like dynamo clones are peers, with all nodes accepting commands, thus merging refers to conflict resolution.

LiveDB behaves like redis with regards to persistence and replication. Antirez weblog (redis author) has a lot of discussion on consistency and availability.

Merging in LiveDB means aggregating query data from multiple nodes. There is no notion of possibly different versions of the same data coming from multiple nodes. Nonetheless, here's the PartitionClient source and some test code demonstrating it's use, scroll down to the test method named PartitionClient.

Also, there are some predefined merges for sorted arrays.

There is no state to be recalled in the event of a network failure. Commands are atomic so there is no transaction to recover, the command was either committed or it wasn't. There is no built in recovery, or way for the client to discover the outcome after reconnecting.

Thanks for the references, I look forward to reading up on the topic.

New Post: My questions

henke — Sun, 11 Nov 2012 15:01:50 GMT

I'm also for developer productivity and my idea of it is that a given database should support the features that make me productive. For example, I'm more productive if I can sleep at night, because I know that my customer's system depends on a single virtual machine to stay up, but rather a cluster. Or I know that my event reactors (things reacting to events) can stay coherent between time-separated data, like in the case of atomic pairwise updates of values (things I'm using Sagas for right now, but would prefer to do in a DB). In general a transaction is not just something implying ACID on a single server, but a way of thinking about a version or consistency boundary for some entity.

It's completely possible to make a database multi-faceted so that I can choose what consistency I want where (as opposed to implementing it manually) - so this is why I'm asking these questions.

You state that result merging is performed by the client; could you give a code example of this?

By "cluster remote engines to have no single-point of failure" I mean e.g. a dynamo-clone like Cassandra or Riak that handles data merging by setting sane W and R values and then doing gossiping, read-repairs to keep the data correctly distributed -- clients still do the merging here though, if there was a partition and we have sibling values. But I guess that you don't have to tackle this, since you have constrained the Server-variant of LiveDB to be a single point in the network topology?

Calls returning; yes I understand that "if no exception is thrown", but how do you regain the call context/transactional context/logical context if a network call fails?

I've read a bit on VoltDB, but I can't say I agree with Stonebreaker; especially his speech at Strangeloop 2012 was a bit of a put-off, with him coming across as a pretty sad person with all the bashing of other databases (in my opinion). With those choices of trade-offs I prefer the transactor-achitecture of Datomic which pushes data to the client in a MVCC fashion that's similar to event sourcing; but in Datomic the aim is provide good querying semantics. The Calvin-transaction paper was also a good read. When doing domain modeling where the aim is to provide a succinct model which doesn't expose *any* properties that can be read/projected onto a view.

I can't say that I have any particular resource in general, but here are a few; Basho's docs, Cassandra's wiki, things from NetFlix like http://www.slideshare.net/adrianco/architectures-for-high-availability-qconsf, things from Spotify, talks by Rich Hickey, papers about transactional OLTP systems in general (Beyond Transactions - An Apostate's Opinion e.g.), the old paper about Sagas, reading up on CQRS and really understanding it, understanding the trade-offs and protocols of RabbitMQ (ACKs to the server, sender, HA-queues etc), things from Boundary, talking with people at different user-groups.

New Post: My questions

rofr — Sun, 11 Nov 2012 04:00:07 GMT

Actually, I mean distributed data store, as in your second example. I mean supporting some sort of ACID in a distributed setting.

Our primary target is as an alternative to RDBMS for OLTP scenarios. The main objective is developer productivity. The horizontal data partitioning feature is an experiment and has no support for distributed transactions whatsover. (We have been discussing it though.)

The nodes of a partition cluster have no knowledge of each other. Query/command dispatching and result merging is performed by the client. As long as each command only targets a single node ACID still holds. When not, only durability is left.

So there's no way to cluster remote engines and have no-single-point of failure and data merging?

Not sure I understand you here...

No way to make sure that writes don't crowd out readers? Surely it would be enough to have lots of writers to make the database completely unusable then?

The default NET ReaderWriterLockSlim implementation has starvation issues, but the behavior is pluggable. That said, we recommend designing for maximum throughput by keeping transactions as short as possible. The leverage gained from running entirely in RAM, using native data structures as opposed to datapage mapping and eliminated lock and latch handling outweighs by far the concurrency sacrificed due to serialized write transactions. Check out the research paper "OLTP through the looking glass", coauthored by Stonebraker of VoltDB.

What does it mean that a client 'call' returns? What if the network message is lost but has its effect? Is the client call then returned despite timing out? When you speak of slave nodes; how do you detect failure of the master and fall over? How do you handle the possibility of faulty failure detectors and merge the resulting state?

By returns I mean if the statement completes without throwing an exception it is safe to assume that the command has been executed and written to the journal. If the connection is lost, an exception is thrown, and there is no way of knowing if the command was successful or not.

We're still working on automatic failover which is to be released with LiveDB Server 0.3b The implementation is based on a quorum formed by the master and two slaves, or the master, a slave and a witness. With a backing SQLStore, manipulating the command journal is fairly simple. We have a command line utility to manipulate FileStore based journals, it needs more features, though.

I mean that there's a difference between a read and write; a write's aim is to persist a business-state-change to disk, while a read is used for taking decisions or displaying aggregations, stats, reports or whatever on, but they are not the same. If you keep the commands as the source of your reads, you'll have the same hairy non-separation between reads and writes that causes ORMs to be painful to use when the model/domain moves on, but the views don't.

You are free to model as you wish with LiveDB. You can define views within the model and update them in response to events coming from the domain objects, all within the scope of a single transaction. Or you can capture events and push them to your enterprise servicebus.

So why does it fall apart?

I wasn't referring to distributed data systems in general, just to horizontal data partitioning with LiveDB.

Do you have any reading references on distributed systems and transactions, CAP, etc to share? Articles, blogs, books, research papers, etc

New Post: My questions

henke — Sat, 10 Nov 2012 22:50:57 GMT

Inline

True, the open source engine runs in-process. LiveDB Server, commercially available, hosts an engine in a rermote process, accepting commands and queries over a tcp/ip connection. Both have full ACID support. When you say distributed, I assume you mean distributed computing (client/server) and not distributed data as in sharding/partitioning over multiple cluster nodes. LiveDB Server has support for both failover clustering and data partitioning.

Actually, I mean distributed data store, as in your second example. I mean supporting some sort of ACID in a distributed setting.

Commands and queries are transactional units. If you wish to have multiple commands per transaction you must define a custom composite command. There is no way to begin a transaction, send multiple commands and then either commit or rollback.

Concurrency, by default, is handled using a ReaderWriterLockSlim allowing multiple readers or a single writer. Thus, transactions have an isolation level equivalent to serialized. There is no difference between an in-process or remote engine.

So there's no way to cluster remote engines and have no-single-point of failure and data merging? No way to make sure that writes don't crowd out readers? Surely it would be enough to have lots of writers to make the database completely unusable then?

When a client call returns, the command has been successfully written to the command journal, and in the case of failover clustering synchronously replicated to each slave node. This is clearly a high availability (HA) setup.

I'm not sure I understand what you mean by projection over the properties of the domain objects. The database (or entire set of domain objects if you so will) is an in-memory projection of the commands executed since the beginning of time. Queries read the in-memory projection directly so there is no distinction between views and domain objects.

I sometimes think of commands as the primitive operations of the domain, that's what I associated with first class citizens. Now that you provided more context to the tweet I'm not sure how this all relates :) With transactions across distributed data (if that's what you meant) ACID pretty much falls apart.

Well, there's atomicity for a reader: that is done by making sure the change of n values are visible at the same point in time. Consistency is garantueed on a per-domain object level in your case, but there's also the consistency as in CAP equating to consistency of logical values - I expect value A and value B to update according to B = 2A; if that is my consistency requirement, I need to make it true, even if that requirement is on distributed data. Or a counter; consistency is that it is the sum of all parts at some point in the future; that's clearly defined and possible in distributed systems, too. So why does it fall apart?

New Post: My questions

rofr — Sat, 10 Nov 2012 18:33:52 GMT

Hi Henrik!

Thanks for taking time to comment.

I hope this alll makes sense!

New Post: My questions

henke — Sat, 10 Nov 2012 08:36:19 GMT

Hi Robert,

I saw your tweet and wanted ensure I understand some things about the runtime of LiveDB.

First of all; as far as I understand, this is a local database - you can't run it distributed with the same durability and isolation semantics as you'd have in memory?

Is a command always a 'transaction' and in that case; what is the multi-threading story; how will data be shared between threads taking and executing commands from the network?

Does it have high availability in a way that allows me to get ACKs on commands that I send from the client library, in a way that an ACK ensures another computer has it and if I go down, that computer will take over?

With regards to querying; is querying a projection over the properties of the domain objects? In that case; don't we have the same problem of keeping views and domain objects in sync that causes so much problems for ORMs?

You replied to a tweet of mine about having atomicity constructs in distributed data stores as a first-class citizens; in particular I was referring to the ability to specify what atomicity I wanted in the updates of two separate values as seen in a view; values that may sourced form different domain entities -- how do commands or proxies solve this?

Henrik

New Post: Discussions are open

rofr — Tue, 22 Nov 2011 21:50:21 GMT

This discussion list is monitored by the team, so bring it on!