Write once file system (WOFS)

eLua write once file system

(v0.9 and above) The WOFS (Write Once File System) is a read-write file system designed to use the MCU’s internal flash as storage. It has an important limitation: a file can be written only once, in a single shot (although there is a mechanism to "overwrite" a file that has already been written, see below for details). After the file is written, it is not possible to append more data to it or modify its current data. While this limitation might seem prohibitive, WOFS can have quite a few practical uses, including:

  • receiving files and saving them in the internal flash using the recv command in the eLua shell.

  • copying a file from another file system in the internal flash using the cp command in the eLua shell.

  • save the history of the code you typed in the interactive Lua shell if linenoise is enabled.

  • data logging. Generally a data logger always appends to its log file, just like WOFS.

Also, WOFS can be completely reinitialized to a "blank" (empty) state using the wofmt command in the eLua shell.

WOFS has a number of important advantages over a "real" read-write file system:

  • very low footprint. WOFS is implemented as a simple extension to the ROM file system, using an internal file system representation which is very similar to the one ROMFS uses. In fact, both file systems are implement in src/romfs.c.

  • it is flash-friendly. The WOFS internal file system structure is completely linear, thus flash sectors are written in natural order from the first free sector to the last sector in flash. This eliminates the need for a flash wear leveling layer.

  • it keeps data in the internal flash and each file occupies a single contiguous block of memory. Since the internal flash is directly accesible by the MCU, this important property allows Lua bytecode files in WOFS to take advantage of some eLua memory optimizations (for example the read-only strings and executing bytecode directly from flash)

  • it needs very little RAM. In a fully blown read-write file system, a flash sector would be split into logical "blocks" used by the files in the filesystem. If the sector must be erased, but some blocks inside it still contain useful information, the file system implementation would need to copy the block content in RAM, erase the sector and write back the useful information in flash from RAM. As some eLua targets have flash sectors which are quite large, this operation might fail due to insufficient RAM, most likely leaving the file system in an inconsistent state.

Also, the WOFS implementation can logically overwrite a file. For example, you can issue this command multiple times:

$ recv /wo/f.lua

After each execution of the recv command, there will be a single /wo/f.lua file, updated with the latest data received by recv. However, the previous "versions" of /wo/f.lua are not actually deleted. Instead, a "file deleted" flag is set on these previous versions, effectively making them invisible to the rest of the system. They are still physically in flash though, so they occupy memory just like a regular file.

Enabling WOFS in eLua

In order to enable WOFS, you need to tell the implementation how much flash the eLua image uses. This information can be made available at compile time by exporting a linker command file constant named flash_used_size. The value of this constant must reflect the total size in flash of the eLua image, including the code, constants, data section and possibly other sections. For an example of how to define this constant, check lm3s.ld in src/platfrom/lm3s.

Another thing that you need to specify is the internal flash structure. The flash memory is divided into contigous areas called sectors (a sector is the smallest erasable unit in a flash memory). The sector organization can vary greatly amongst various MCUs, but eLua provides a generic mechanism to describe the flash structure:

  • if the flash is divided into equally sized sectors, define the INTERNAL_FLASH_SECTOR_SIZE macro to the size of one flash sector.

  • if the flash sectors have different sizes, define the INTERNAL_FLASH_SECTOR_ARRAY macro as an array that contains the size of each flash sector in turn.

Check your MCU datasheet to find which of the above variants you need to use.

Other macros that you need to define are given in the table below:

Option Meaning

INTERNAL_FLASH_SIZE

The size of the internal flash memory, in bytes

INTERNAL_FLASH_START_ADDRESS

The start address of the internal flash memory in the MCU address space

INTERNAL_FLASH_WRITE_UNIT_SIZE

The unit size of the Flash write access routine (see below).

Finally, your platform implementation needs to define two functions for accessing the internal flash. The first one is a flash write function, called by WOFS to actually write data in flash. Its signature is in inc/platform.h:

u32 platform_s_flash_write( const void *from, u32 toaddr, u32 size );

Depending on your platform, the flash write hardware may have different requirements. Some MCUs only allow writing the flash in multiples of a power of 2 (usually 4), at an address which is a multiple of a power of 2, or both. If this is the case for your MCU (check the datasheet), define INTERNAL_FLASH_WRITE_UNIT_SIZE to the required alignment. This way, you can be certain that your platform_s_flash_write function will be called only with a destination address (toaddr) that is a multiple of INTERNAL_FLASH_WRITE_UNIT_SIZE and with a size (size) that is also a multiple of INTERNAL_FLASH_WRITE_UNIT_SIZE.

The second flash-related function is used to erase pages from flash (used only when "formatting" the WOFS image via wofmt, as already explained). Its signature is also in inc/platform.h:

int platform_flash_erase_sector( u32 sector_id );

Check this link for more details about the flash platform interface. If all the above requirements are met, just define BUILD_WOFS in your platform_conf.h file, compile and burn your eLua image and you’ll have a WOFS instance up and running. Check here for more details about the configuration of your eLua image.

Using WOFS

Important the examples in this section are written in Lua, but the exact same observations apply for C code that needs to work with files in WOFS.

The WOFS is mounted under /wo, so in order to open a file from WOFS simply prefix its name with /wo/, as in the example below :

f = io.open( "/wo/f.lua", "wb" )

Don’t forget the fundamental limitation of WOFS: once you open a file in write mode, you need to write the whole content of a file before calling f:close(). After you close the file, you won’t be able to open it in write mode anymore. Also, if the file is opened in write mode, you’ll only be able to write data at the end of the file. You can still use f:seek to read data from any location in the file, but if you try to write when the file pointer is not positioned at the end of the file you’ll get an error. You can use the append mode to automate this process. If a file is opened in append mode, its file pointer is automatically positioned at the end of the file before each write operation. To open a file in append mode, simply use a instead of w when opening the file:

f = io.open( "/wo/f.lua", "ab" )

The last things that you need to remember: always close your WOFS file descriptor when you’re done with it. This is extremely important. If the file is opened in read mode, forgetting to close its file descriptor will just leak some memory, which is a non-fatal error. However, if the file is opened in write mode (this includes append mode) and you foget to close its file descriptor, the file size will not be correctly registered in the WOFS internal structure, which will lead to a corrupted file system. So remember to do this:

f:close()

This isn’t actually such a big problem with Lua, since a file is automatically closed when the file’s userdata is garbage collected, but becomes very important when using WOFS in C code.

Notes

Some things you should consider when using the WOFS:

  • WOFS shares the same memory that is used to hold the eLua firmware. Although WOFS shouldn’t touch the eLua firmware, various bugs in the code might render eLua unusable. In this case, simply reflash eLua and everything should be fine.

  • there is currently no way to find how much free space exists on WOFS. If you write data to WOFS, always compare the length of the data being written with the length reported by the write function (the actual number of bytes written). If they are different, this most likely means that there is not enough data left on the internal Flash.

  • while in theory it is possible to sometimes keep the WOFS contents after reflashing the eLua firmware, this is a manual, error prone procedure that will not be described here. Better keep in mind that after reflashing the eLua firmware your WOFS will also be initialized (empty). So remember to save all the important files in WOFS before reflashing the firmware.

  • WOFS is not a robust file system. If a power failure happens when a WOFS file is opened in write mode, a file system corruption is almost a certainty. The same holds true in the case of a serious error in the eLua code that triggers a non-recoverable CPU exception, or if you simply hit the RESET button on the eLua board by mistake. As a general rule, do not use WOFS to store important data.