[Yaffs] ECC byte ordering (Possibly: SIGH, AGAIN?)

[Steve Wahl]

Hello, everyone.  I'm new to this mailing list, but I have attempted
to read the archives for answers to this.  I'm not coming up with
anything definitive that way, so I thought I'd email you.

Background, indented so you can skip if you want to:

     The company I work for has a series of Linux 2.4 based products,
     using YAFFS on nand flash.  Somebody else did the port of YAFFS
     to 2.4.  They are no longer around (but I think they did an OK
     job of it as far as I can tell).

     I'm making a port of linux 2.6.9 to the same hardware.  It's
     intended that eventually an upgrade of the software on the units
     will upgrade the kernel version.  (And 2.6.9 might not be the 2.6
     we stick with, of course.)

     I had to choose whether to just move the yaffs version we were
     currently using forward to 2.6 myself, or go with a more current
     version of yaffs in which others have probably ironed out most of
     the kinks with 2.6 -- so I of course chose the latter.

     I used the patches in the archive from Frank Rowland.  (Thanks,
     Frank!)  I saw that Charles Manning had a few misgivings about
     the method, but it seemed the faster way to get my tree up and
     running, and then later I could compare with CVS for any
     differences.  (I also found the CVS tree seemingly lacking in the
     "how to put it in the kernel" area -- perhaps it's in the middle
     of being updated, and I came to the party at exactly the wrong
     time?)

     I had to make some new choices in the config file, specifically,
     YAFFS_USE_NANDECC.  The help says, if unsure, say "Y".  (I now
     see that this config variable WAS present in the older kernel, I
     don't know why I slipped into changing it.)

Short story is, I ended up using MTD's ecc in 2.6, where the old 2.4
kernel was using yaffs's ecc.  I was booting both kernels on the same
hardware, with the same flash chip; each version didn't care much for
what the other left in the NAND chip.  Of course, the order of the ecc
bytes was wrong, causing me no end of headaches searching for what I
did wrong.

Could I humbly suggest a patch like this:

----------------------Cut here--------------------------------------
--- yaffs_mtdif.c.orig	2005-02-09 10:02:47.000000000 -0600
+++ yaffs_mtdif.c	2005-02-09 10:05:19.000000000 -0600
@@ -35,7 +35,14 @@
 // this is for versions of mtd nand driver in kernel 2.6.8 and later
 	eccbytes: 6,
 #endif
+
+#ifdef CONFIG_YAFFS_ECC_WRONG_ORDER
+	// The mtd nand code calculates the ECC bytes in a different
+	// order than the smartmedia standard
 	eccpos: {8, 9, 10, 13, 14, 15}
+#else
+	eccpos: {9, 8, 10, 14, 13, 15}
+#endif
 };
 
 struct nand_oobinfo yaffs_noeccinfo = {
----------------------Cut here--------------------------------------

So that the byte order of the ECC in the binary filesystem doesn't
change when CONFIG_YAFFS_USE_NANDECC changes state?  It seems to me
that the order of the ECC bytes belongs to the filesystem standard,
not the underlying layer -- in other words, every yaffs filesystem
should have the same ECC byte layout.

OR, failing that, maybe we could change the following portion of the
2.6 Kconfig (I found this in Frank's patches, but couldn't find it in
the cvs tree?)

----------------------Cut here--------------------------------------
 +config YAFFS_USE_NANDECC
 +	bool "Use ECC functions of the generic MTD-NAND driver"
 +	depends on YAFFS_FS
 +	default y
 +	help
 +	  This enables the ECC functions of the generic MTD-NAND driver.
 +	  This will not work if you are using the old mtd.
 +
 +	  NB Use NAND ECC does not work at present with yaffsram.
++
++	  NB Use NAND ECC has a different ECC byte order than the
++	  normal yaffs_ecc.c, and the resulting filesystem format is
++	  binary incompatible with a previously existing yaffs filesystem.
++	  (This different ordering is the same as yaffs_ecc.c generates
++	  when ECC_WRONG_ORDER is turned on.)
 +
 +	  If unsure, say Y.
 +
 +config YAFFS_ECC_WRONG_ORDER
 +	bool "Use the same ecc byte order as Steven Hill's nand_ecc.c"
 +	depends on YAFFS_FS
 +	help
 +	  This makes yaffs_ecc.c use the same ecc byte order as
-+	  Steven Hill's nand_ecc.c. If not set, then you get the
-+	  same ecc byte order as SmartMedia.
++	  Steven Hill's nand_ecc.c (part of the MTD-NAND driver).
++	  If not set, then you get the same ecc byte order as SmartMedia.
 +
 +	  If unsure, say N.
 +
----------------------Cut here--------------------------------------

(That last diff was hand-generated in my editor here, just to give you
the idea.  You may have problems trying to apply it directly...)

I'm sure things aren't this simple, that of the current shipping
implementations of YAFFS with MTD out there, there's probably enough
existing yaffs filesystems in each camp (Steven Hill/MTD ECC byte
order vs. YAFFS ECC byte order), that there's no way everybody gets
out of this with nobody feeling any pain.  

But it seems to me that yaffs filesystems outside of those paired with
MTD in linux would have an ECC ordering matching (YAFFS_USE_NANDECC=n,
YAFFS_ECC_WRONG_ORDER=n), so that should be considered the "correct"
ordering.  And therefore, changing the configuration so people
currently using (YAFFS_USE_NAND_ECC=y, YAFFS_ECC_WRONG_ORDER=n) have
to change to (YAFFS_USE_NAND_ECC=y, YAFFS_ECC_WRONG_ORDER=y) when they
update their code doesn't seem like too much pain to get everybody on
the same page, does it?

Thanks for patiently listening to a newbie!

--> Steve Wahl