README.md
1This directory contains files related to storage of read-only S2 data in blocks.
2The classes are generic.
3
4The data file format is intended to be language neutral (e.g. Java or C code could easily be
5written to read it with minimal dependencies), providing a good balance between file size,
6extensibility and memory usage at runtime.
7
8High level file structure
9=========================
10
11 * `src/readonly/` - host + device code for reading data files
12 * `src/write/` - host code for writing data files
13 * `src/test/` - host tests for readonly/ and write/ code
14 * `tools/` - host tooling to support generation / debugging / testing of data
15 files.
16
17Developed for Android, the code is also intended to work with host Java for easy
18testing and debugging. The code is split into "readonly" and "write" code, as
19the code that writes the files is expected to only be used on host during file
20generation, while the "readonly" would be used on both host and device.
21
22Block file format information
23=============================
24
25A "block file" is a general-purpose file format containing a small amount of header information,
26blocks of data, and metadata about those blocks. All types are big-endian / network byte ordered.
27
28Blocks are assigned contiguous IDs which are numbered from zero.
29
301. The file header has a type-identifying magic, and a version.
312. Next are the block infos, which hold metadata about all blocks in the file such as their ID,
32a type ID, (optional) arbitrary "extra" bytes, and size / offset information for the block.
333. Lastly, come the blocks of data themselves. Blocks can be zero-length, in which case they take up
34no space in the file.
35
36Packed tables
37=============
38
39Packed tables are a way of arranging block data to store tables of key-ordered key / value pairs in
40a compact way. Each entry in the table takes up a fixed number of bytes.
41
42Packed tables may contain some (optional) shared information that applies to all records in the
43table. Then they contain one or more fixed length `{key}`/`{value}` records of `{R}` bits sorted by
44`{key}`. The table data is easily memory mapped and each record can be randomly accessed by
45`{key}`.
46
47