File Stability
It is quite common to poll a directory for changes and then do something âinterestingâ with new files. But as one process is writing to a file, other processes have no idea when the file has been fully written. One solution is to add a second âindicatorâ file that we create after creating the first file. This is intrusive, however.
There is a neater way, which is to detect when a file is âstableâ (i.e., no one is writing to it any longer). FileMQ does this by checking the modification time of the file. If itâs more than a second old, then the file is considered stableâat least, stable enough to be shipped off to clients. If a process comes along after five minutes and appends to the file, itâll be shipped off again.
For this to work, and this is a requirement for any application hoping to use FileMQ successfully, do not buffer more than a secondâs worth of data in memory before writing. If you use very large block sizes, the file may look stable when itâs not.
Get ZeroMQ now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.