>I see how you know :-)
Heh, yeah, I should have mentionned it was a shameless plug.
OK, I have done some investigations, and here is the problem. The 2.2
or 2.4 MB occupied it shows isn't caused by a bug, it's a feature
(maybe not fortunate, though).
What happens is that the system creates objects of say 800 bytes. For
such blocks, since it's bigger than 510, I get a free sector if
available (otherwise, I get the largest one), and then for the
remaining 800-506 = 294 bytes (there is a 2 bytes header plus 4 for
next segment ID), I put it in a new sector as well (since you nearly
don't have any used sector when you add the package). And so on for
every 800 bytes objects (since 294 bytes plus 2 for header won't fit
in a sector where I already have 296 bytes).
In the end, we have something like 20% of unused space. This doesn't
explain totally the 2.2 MB, there is another cause.
When the system created an object with a zero size, I try to get it
an empty sector, just because the system will very likely enlarge it.
Normally, when I put a sector in transaction, I get another sector to
have a copy of it to abort the transaction if it fails (and
therefore, I can restore the store to the previous state), and this
is freed when the transaction is over. But when I put a free sector
in transaction, I don't need to do so, because the original sector
was empty, I just write so in the transaction table. So this way, you
can do a big transaction to create a big object if you have plenty of
space on the card (just as you do with your package, which, although
it's 1.3 MB, takes 2.2 MB in sectors if you add it on an empty store
which is more than 2 MB).
But, when you want to delete it, the original sector is used, and
freeing it means getting a free sector, copying to it and then
freeing the original object. Copy sector is marked as non data to be
not used (and is of course no longer free), but first sector isn't
marked as free (or whatever the new used size of it is) until the
transaction is over.
So at this point, we needed something like 2 x 4400 free sectors
while we only had 3600 left. And of course it failed.
Of course, I have to figure out why the transaction doesn't properly
reverted to the previous state, but I have logged entirely the
deletion of the object (it took more than 4 hours and the log is
about 11 MB long, larger than the previous largest log I made, still
when debugging ATA Support, 5.8 MB).
Nevertheless, I also need to find a way to prevent this to happen. I
don't know if I really can change the written size of sectors when
doing transactions, it might lead to big problems. Let's say that I
do, it's possible nevertheless that you won't be able to delete an
object you succeeded to store. I can also have plenty of problems if
you do revert a transaction on a single object, which the system does
in some particular cases, and so on.
I also thought this morning before I found out the cause of your
problem that I should reserve a pool of sectors for transaction when
the store becomes full. But I surely can't reserve 4400 of them. I
don't know how many to reserve, though, probably something depending
on the size of the store.
In the end, I start to wonder if (a) linear stores do transaction on
commands i.e. store a list of operations and execute them when you
commit and (b) if they only do transactions on meta data (i.e. where
objects are).
(a) and (b) are incompatible with my wish of doing the maximum for
the safety of data. Indeed, if a sector cannot be written for any
reason, the whole transaction will be cancelled with the current
implementation. If I do store the journal of commands to execute, I
can't do that.
And I don't think that (b) is the case considering the APIs, although
I have doubts.
Paul
-- Home page: http://www.kallisys.com/ Newton-powered WebServer: http://newt.dyndns.org:8080/-- This is the Newtontalk mailinglist - http://www.newtontalk.net To unsubscribe or manage: visit the above link or mailto:newtontalk-request_at_newtontalk.net?Subject=unsubscribe
This archive was generated by hypermail 2.1.2 : Thu Nov 01 2001 - 10:02:22 EST