Rework the pathways serialization format

[MaybeJustJames]

The "current" pathways serialization format (what I've been calling "abomination") is a custom format that requires serializer/deserializer modification to extend.

Issue #25 perhaps requires extension of the format. Rather that extend the format, this issue proposes replacing the custom format with a standard serialization format.

Some options to consider (also a paper):

Serialization format	Human readable	Appendable	Multi-language impls	Standardised	Extensible	Compact
Abomination	✅	✅	❌	❌	❌	❌
YAML	✅	✅	✅	❌	✅	❌
JSON	✅	❌	✅	✅	✅	❌
SQLite	❌	✅	✅	❌	✅	✅
JSONlines	✅	✅	✅	❌	✅	❌
CBOR	❌	❓	✅	✅	❓	✅
Protobuffers	❌	❓	✅	❌	❓	✅
Flatbuffers	❌	❓	✅	❌	❓	✅
Avro	❌	❓	✅	❌	❓	✅
Thrift	❌	❓	✅	❌	❓	✅
Cap'n'proto	❌	❓	✅	❌	❓	✅
Twine	❓	❓	❓	❓	❓
Preserves	❓	❓	❓	❓	❓	❓
UBJSON	❓	❓	❓	❓	❓	❓
Postcard	❓	❓	❓	❓	❓	❓

Human readable

Plain text encoding.

Appendable

I don't need to know about more than the single row I'm appending to the file in order to append (e.g. JSON is not appendable because of array delimiters).

Multi-language impls

There are off-the-shelf serializers/deserializes for the format in Python and at least 1 other language.

Standardised

The format is documented in an internet standard from IEEE, W3C, etc.

Extensible

When a field is added, old software can still work with data serialized with the new field.

Why?

I think there are a few good reasons to consider this change.

1. Current format requires specialised knowledge to understand the data format itself (not just domain knowledge of metabolomics). Using a more common format means that someone receiving the data can use an off-the-shelf parser and be confident that it works.
2. eval()-ing Python is slow and dangerous. literal_eval() is safer but still much slower that parsing, say, YAML. Also opens up the possibility of using the data in non-Python languages. I'm already doing this a little in the command-line client app I'll give you which is written in Rust. Parsing literal Python values is painful but possible outside of Python and using a more common format makes this much easier.
3. Extending the data you want to store becomes much easier. You don't have to make fundamental adjustments to the format to add a citation field. In YAML you would just add an optional key to each object in the list that has a citation. In sqlite you'd add an extra nullable column. Both these options are less data than a 3 byte empty list for each entry in your custom format (~35k for a single 12000 entry file).

[MaybeJustJames]

I've run some tests to find out if the abomination format of storing metabolite labeling patterns saves any storage at all.
Take a simple example with the following labeling patterns that need to be stored:
['0__', '1__', '__']
Using the current scheme, this would be stored as:
('0(1/3),1(1/3)', (('0', 1), ('1__', 1)), 'T')
I wanted to know if explicitly laying out all possible labeling patterns would take more space, so I tried to store this example like this:
(['0__', '1__', '__'], True)

Naturally this example is a lot shorter but maybe not representative. So I tried to re-encode all of the files I have at the moment with this scheme (the only change being how metabolite labelings are serialized) and here are the results:

• ALPHA-GLUCOSE__L-LACTATE__30 *
  yours: 112492654
  mine: 100328739
  mine is ~11% smaller

• ALPHA-GLUCOSE__UMP__30__pp09 *
  yours: 1231057202
  mine: 1182305592
  mine is ~4% smaller

• GLN__PRO__10__pp09 *
  yours: 6882337
  mine: 4945749
  mine is ~28% smaller

• PYRUVATE__RIBULOSE-5P__25__pp09 *
  yours: 666681344
  mine: 608105319
  mine is ~9% smaller

So from this sample, the explicit encoding is consistently smaller. I haven't checked if each individual (per line in the file) encoding is always smaller though.