8 bit os_f handles

[John Tytgat]

In message <200003300157.RAA06879 at purple.trimedia.sv.sc.philips.com>
          "Jonathan Coxhead" <jonathan at doves.demon.co.uk> wrote:

>    John wrote,
> 
>  | The 32 bit filehandles can come via another path : today,
>  | somebody can enhance the sequential file access by intercepting all
>  | filehandle related operations and let this util return >255 value
>  | filehandle (e.g. ptr's to his workspace per open file).  And at that
>  | point, all our OSLib based programs might go down because of his
>  | perfectly legal & correct piece of software.  That's what I want to
>  | avoid.
> 
>    I betcha there are no such things, and (if anyone's tried to make 
> them) they didn't work.

Are you challenging me ? Note I said >255 value filehandles, not
"more than 255 filehandles".

>    Remember, FileSwitch keeps track of *all* file handles that are 
> issued by *any* filesystem. As I recall, it only keeps 8 bits of 
> information about each one, so using more is not possible.

I'm not talking about extentions allowing more than 255 filehandles,
I'm talking about extentions giving you filehandles with a value
bigger than 255.

>    I'm still not convinced that we will see any product, that is 
> recognisable as "FileSwitch", that has 32-bit handles. My guess would 
> be that in the course of such a major rewrite, many other 
> incompatible changes would get made.

Like ? As I said before, the RISC OS PRM's do not declare filehandles
as 8 bit values.

> A change to a 32-bit handle 
> interface in OSLib, is a change to support what is, at the moment 
> (and with the best will in the world), "vapourware".

Not correcting this bug makes the people's life @ RISCOS Ltd/Pace
more difficult.  If they want to issue filehandles with values
bigger than 255, they will receive complaints of people seeing
problems with OSLib compiled programs.  Who gets blamed ? The
wrong ones IMHO.

>    If I still had anything to do with making OSLib decisions, the 
> only thing I would do at the moment is wait to see what actually 
> appears. I expect that there are going to many compatibility problems 
> with upgrading to a 32-bit FileSwitch, and the handle width will be 
> the least of them.

Please give examples.

>  | >    That's not true---they will only fail to run if they get a file 
>  | > handle that is actually >255. This might never happen.
>  | 
>  | Euh, why "never" ?
> 
>    People who don't get the new FileSwitch will never get handles 
> with more than 8 bits. Why should they have to go to the trouble of 
> upgrading, for something they don't need? This is more Microsoftish 
> philosophy.

But as a programmer, I want my programs to be run on as many versions of
RISC OS as possible, the old ones and future new ones and I do want to
maintain only one version, i.e. the one assuming filehandles 32 bit wide.
Therefore I want OSLib to treat filehandle as 32 bit values like defined
in the PRMs.

This discussion isn't about users to be forced to upgrade.  It is about
making sure that today's OSLib compiled programs keep on running on
future versions of RISC OS which have filehandles bigger than the 255
value.

>  | I see two scenario's where today's OSLib based
>  | programs can fail because of this os_f 8 bit issue :
>  | 
>  | 1. A new version of RISC OS with upgraded FileSwitch returning >255
>  | value filehandles.  All OSLib based programs we link today might
>  | break on that OS.  That's something I want to avoid to doing the
>  | necessary fixes today.  It is dummer to hear that your programs
>  | don't run on RISC OS x because you were using a library incorrectly
>  | making assumptions about something.
> 
>    To fix those problems, even the compatibility-breakers admit you 
> have to recompile your code. If you're recompiling it, you can easily 
> upgrade to |os_w|.

It is easier to recompile and test the code (eventually also source
eyeballing) than to change any instance of os_f into os_w and to recompile,
test, etc.

>  | 2. Someone writes a module enhancing sequential file access and
>  | remap's FileSwitch handles by its own handles which have values
>  | bigger than 255.
> 
>    This won't work, for reasons described above.

Rubbish.

>  | If we act now by changing os_f to a 32 bit wide value, all programmers
>  | can recompile their programs now, retest it and spread it before one
>  | of the two above mentioned scenario's happen.
>  | 
>  | > and possibly the sources have been lost.
>  | 
>  | If the sources are lost, then I don't see any point in relinking
>  | with a new OSLib.  I suppose development of that product is dead. Do
>  | we really have to consider this case ?
> 
>    Of course. Most modules are distributed in binary form.

Which OSLib based modules without source are around ? I don't know a
single one ?!

>  | > We can't force people to recompile their code, any more than we can
>  | > force them to change |os_f| to |os_w|.
>  | 
>  | Of course we can not force people as they are not forced to use
>  | new OSLib versions.  But we can gently say : "If you want
>  | to use this version, note that os_f ... etc.".
> 
>    But why should we want to do this, when there's a perfectly simple 
> upgrade path that doesn't require us to browbeat people into doing 
> things in a new way?

- To keep things as simple as possible in OSLib.
- To make sure that users of the OSLib don't be tempted to use the 8 bit
  filehandles ever.
- To stop the idea that there is such a thing as an 8 bit filehandle.

> [...]
>  | And programs relying on the fact that os_f is 8 bit and always will be
>  | 8 bit and go wrong when os_f gets a size bigger than 8 bit or are used
>  | to story anything else than RISC OS filehandles, are buggy and should
>  | be fixed ASAP.
> 
>    The goal should be to support the correctness of existing correct 
> programmes, and to allow new correct programmes to be written.

The issue is that an OSLib based program using os_f today is *not*
correct and might fall over when encoutering filehandles with value
bigger than 255.  So your statement does not apply for this os_f issue.

>  |   1. Are RISC OS filehandles 8 bit or 32 bit ?
>  | 
>  |      My argument in above quoted paragraph concerned this discussion.
>  |      I want to make sure that my OSLib based programs are garanteed to
>  |      work when >255 value filehandles are being used.  I want to act
>  |      proactive on this issue and have a solution.
> 
>    In what way would changing to |os_w| fail to be a solution?

Why the overhead ? Keep things as simple as possible in OSLib.  Don't
tempt the users of OSLib to ever use 8 bit filehandles again as it
is not future proof.  Etc.

>  |   2. How are we going to address this ? By introducing os_w or
>  |      upgrading os_f to 32 bit value ?
>  | 
>  |      This discussion seems to turn around the question of relinking is
>  |      enough or that a recompile based on Makefile is ok or full
>  |      recompile is needed, wanted, realistic, etc. My statement in
>  |      above mentioned paragraph was not meant to be used in this
>  |      discussion ;-). Whatever option is taken for discussion 2, an
>  |      action of the programmer is needed (relink/recompile/whater) and
>  |      that's something I want to do ASAP.
> 
>    It is easy to construct a scenario where a module (for which you 
> have no source) asks for a file handle, and your code writes an 
> |os_f| to that memory location. The module knows that |os_f| is 8 
> bits, and has only given you a |char *| to write through. Your 
> programme thinks |os_f| is 32 bits, and trashes the module's 
> workspace.
> 
>    And it's worse than that---the module interface documentation 
> might have been written using OSLib type names. It could easily say 
> that the pointer is of type |os_f *|. You write an |os_f| through the 
> pointer. Bang!! Whose bug is this? It's not in the module---it 
> described everything correctly, as at the time it was written.

Of course not.  Filehandles are not 8 bit values, not yesterday, nor
today, nor tomorrow.  That interface is broken.  Very though situation
I admit, but it is broken.

> It's 
> not in your programme---how could it know that the module was written 
> before March 2000 (or whenever)?
> 
>    The bug is OSLib's!---if we let it happen. We don't need to 
> construct scenarios---if we guarantee binary compatibility, we are 
> sure that those problems can never arise.

You're sticking your head in the sand.  It is an OSLib problem from
the beginning.  Don't waste time by inventing clever mechanisms which
do not correct the bug but cycle around it.  Those modules will break
in the future anyway.  The modules and their interface need to be
updated.

>    In the example above, the module interface will say that the 
> pointer it is giving you is an |os_f *|. Your code will be writing an 
> |os_w| to it. Bang! Compiler error! Another mistake caught at compile 
> time. Joy! Bunnies hopping in the fields! Etc.

Again with appropriate warnings on the next version of OSLib, the
programmer will do a full recompile and there are no bunnies to
be shot at all.  If he has only modules without source, sorry, no new
OSLib version for him unless he want to play russian roulette.

These modules are anyway broken as they are playing with 8 bit
filehandles which are not future proof so the best he can do is not
stick his head in the sand and ask new versions of his binary modules
right away before we start seeing >255 value filehandles.

>  | However I would advocate to go for the short pain and inform in
>  | the next release of OSLib that this change has happened and that
>  | there are consequences of using this release (with detailed
>  | description). And in that new version of OSLib there is no place
>  | or an 8 bit filehandle.  Sorry but such a thing does not exist
>  | in RISC OS PRM's.
> 
>    This remark is not aimed at you specifically, John, but sometimes 
> it seems that there is kind of hacker who revels in being able to 
> make fools of people. They are happy with being able to answer 
> questions like, "why is my memory being unexpectedly corrupted" with 
> an answer like, "that's because you're using an old version of the 
> library, and I changed it in an incompatible way, so you have to 
> recompile". If the user then asks why the change was made, the hacker 
> can show off his knowledge by saying, "because FileSwitch has been 
> upgraded to use 32-bit file handles". The user then says, "but I'm 
> still using RISC O S 3.5, with the FileSwitch in ROM". The hacker 
> feels smug, because h has been proved "right", and the user "wrong".

But the programmer needs to make programs which can run on RISC OS 3.5
*and* a RISC OS version where we have filehandle values bigger than
255 ! If the programmer didn't read the warnings that from now on
os_f is 32 bit and that all libraries using OSLib needs to be recompiled,
then he didn't do his job properly.

Nobody would let the 'hacker' (in your story) go away like that.

>    Personally, I find it more rewarding, and technically challenging, 
> to be able to supply code that never gives rise to this sort of 
> question in the first place. Any code that has been written and 
> tested will continue working in the same way. The library is a 
> completely stable base on which reliable code can be written.

You're absoletly right when it concerns upgrades for new RISC OS API
entries but we are now talking about a bug in OSLib ! I do not like
to see backwards compatibility with a broken situation.  I do not
like to see new users of OSLib to be tempted to write programs
which are broken per definition.  They may not see it today, but
their programs will break tomorrow.

>    Going for "the short pain now" in even 1 area then allows the 
> potential for other "short pains" in future. Very soon, you have 
> overlapping works in progress with different expectations about what 
> means what, and life descends into chaos. "Chaos bad!", as Buffy 
> would say.

Words, words, words.  Sorry I don't see your point.  Letting os_f live
forever as 8 bit is asking for problems in the future too.  How are
you going to make sure that people aren't going to use os_f in
new code ? If they do, then with >255 value filehandles, we will
see "short (?) pains" all around.  Is your scenarion then so much
better ?

The sad thing will be that the people who are providing >255
value filehandles (RISCOS Ltd, Pace, any third party making filehandle
extentions) will be blamed instead of the programmer wrongly using os_f
in his OSLib based programs.

>  | >    Now, how about font handles ... do we also need |font_w|? 
>  | > Including font_find_font_w(), font_lose_font_w(), font_paint_w(),
>  | > etc, etc, etc??
>  | 
>  | Upgrade to 32 bits and no _w please... <ducks> :-)
> 
>    I just *knew* someone would say that!! :-) :-)

Of course... :-)

John.
-- 
John Tytgat, in his comfy chair at home                                 BASS
John.Tytgat at aaug.net                             ARM powered, RISC OS driven