xput comparative 1.19 vs. 2.0

Here is the comparative for the code in git. I will publish beta2 in a couple of days. As you can see the xput improvements are considerable.

1.19
-----

16 bits on CLUT profiles                     : 10.6667 MPixel/sec.
8 bits on CLUT profiles                      : 9.75015 MPixel/sec.
8 bits on Matrix-Shaper profiles             : 3.8638 MPixel/sec.
8 bits on SAME Matrix-Shaper profiles        : 4.28495 MPixel/sec.
8 bits on Matrix-Shaper profiles (AbsCol)    : 10.4507 MPixel/sec.
16 bits on Matrix-Shaper profiles            : 3.92349 MPixel/sec.
16 bits on SAME Matrix-Shaper profiles       : 3.96924 MPixel/sec.
16 bits on Matrix-Shaper profiles (AbsCol)   : 10.6667 MPixel/sec.
8 bits on curves                             : 4.33839 MPixel/sec.
16 bits on curves                            : 4.3944 MPixel/sec.
8 bits on CMYK profiles                      : 4.09626 MPixel/sec.
16 bits on CMYK profiles                     : 3.96924 MPixel/sec.
8 bits on gray-to-gray                       : 24.3902 MPixel/sec.
8 bits on SAME gray-to-gray                  : 24.3902 MPixel/sec.


2.0
----

16 bits on CLUT profiles                     : 10.1394 MPixel/sec.
8 bits on CLUT profiles                      : 10.6667 MPixel/sec.
8 bits on Matrix-Shaper profiles             : 26.2726 MPixel/sec.
8 bits on SAME Matrix-Shaper profiles        : 30.1318 MPixel/sec.
8 bits on Matrix-Shaper profiles (AbsCol)    : 10.5541 MPixel/sec.
16 bits on Matrix-Shaper profiles            : 10.2433 MPixel/sec.
16 bits on SAME Matrix-Shaper profiles       : 9.84615 MPixel/sec.
16 bits on Matrix-Shaper profiles (AbsCol)   : 10.6667 MPixel/sec.
8 bits on curves                             : 30.0752 MPixel/sec.
16 bits on curves                            : 35.3201 MPixel/sec.
8 bits on CMYK profiles                      : 4.19727 MPixel/sec.
16 bits on CMYK profiles                     : 4.26667 MPixel/sec.
8 bits on gray-to-gray                       : 40.9207 MPixel/sec.
8 bits on SAME gray-to-gray                  : 40.9207 MPixel/sec.
8 bits on CMYK profiles                      : 4.3573 MPixel/sec.
16 bits on CMYK profiles                     : 4.55192 MPixel/sec.
8 bits on gray-to-gray                       : 39.312 MPixel/sec.
8 bits on SAME gray-to-gray                  : 40.9207 MPixel/sec.

Linking tags

Implementing the tag link feature has been easier than I originally thought, but there are some caveats.

lcms2 does support some new features on read/writting profiles. You can use cmsReadTag/cmsWriteTag to read/write lcms objects like LUTs, tone curves and so. You can also use cmsReadRawTag/cmsWriteRawTag to read or write whatever you want, but poor library does not do any checking or understanding on whats going on.

You can also link tags to items created by any of those two methods. So far so good.

But wait, you can also write plug-ins to add more "understood" objects in cmsReadTag/cmsWriteTag and also you can write plug-ins to add new types for those objects.

In addition to all that, there is a brand new structure added as an addedum to ICC spec 4.2, that is the MPE or multi profile elements. This is worth of several comments in this blog. Basically that may make photographers and precission yonkies very happy as it includes *true* floating point numbers, among other things. There is a plug-in type devoted to MPEs. More to come.

So, if you consider all all those acess methods should be consistent, we have a nice mess. The good news are it seems to work, the bad news, we need more testing. But overall I see progress.

Fear not!, maybe I will even accomplish the schedule and realease the whole thing on November.

ZOO Doppelgänger and the Link feature

To some extent, old'n'good lcms allowed profile editing. That was a "dangerous" feature in the sense people may abuse of it to grab copyrighted material. That is, you may open your favourite profile to remove the copyright tag, and here you go. Obviously preventing that by the hard way is just like killing the messanger, so this feature is working in lcms and is up to you to use -or abuse- it.

But reviewing the previous entry about the zoo, I had a wild idea: if the zoo test reads every single tag, what if then I try to rewrite all those tags? that would create a Doppelgänger version of every profile in the zoo, but not necesarely with same organization and size.

The test code was written in few minutes, and after I run it, poor lcms2 get into hyperspace and crashed badly.

Ok, five bugs ahead, I got the writting/copying feature working. But this also has unveiled how profile vendors abuse of the link feature. That is, since a tag is described by some block in the file, I can put two or more different entries in the tag directory pointing to the same location. Humm.. I have to add some code to deal with this case.

Profile ZOO

The serialization part is now complete. That is, lcms2 should be able to read all tags it understands on all profiles in the wide world. Since it understands all tags that are, or have been part of any ICC spec, present or past, a lot of profiles should be readed by current code.

I have been lately worried about stability and qualification. If lcms2 want any success, it should be tolerant with ill-formed profiles. I say tolerant, not permissive, because crafted profiles may be used by the bad guys to introduce exploits. We don't want wargames again, right?

I have compiled along the +10 years of lcms life, a "Zoo" of profiles. This collection includes actually about 1500 assorted profiles, going from the widely distributed v2 sRGB to rare corner cases, like devicelinks holding 12-ink separations. Not to be very common.

Some of those profiles are broken. Well, not completely broken. They have slightly malformed tags, like colorants using a bad type, descriptions with wrong char count, bad sizes in the header...

To check how well lcms2 may deal with that, I wrote a small program that runs across every single profile in the Zoo and then reads every tag in the profile. The code should reject the unuseable tags and behave nicely if some information can be recovered. It cannot in anycase segfault or leak resources.

So, keeping my fingers crossed, I executed the program and ... almost! one segmentation fault. Ok, it was a bug, fixed. Run it again and .. success! no memory leaks, lots of tags discarded and a cool "all is ok" printf'd at the end. Pfew!

The probe profile

"Jimmy Volatile" a Lcms user, suggested to incorporate test plots for the regression tests. In this way external apps using lcms could check if all is working as expected. I think this is a good idea, and maybe it is also feasible (all depends on the schedule).

An interesting check would be to use the ICC probe profile. This comes from the ICC site:

"The 'probe profile' (Probev1_ICCv2.icc) is syntactically a v2 ICC output device ('prtr') profile, and can be used in a workflow wherever such a profile is required. The color space of this profile is CMYK, and its PCS is Lab.

Colors processed via this profile are deliberately distorted in a systematic way, to enable visual determination of the rendering intent used when rendering ("BToA" or PCS to device transforms) and when proofing ("AToB" or device to PCS transforms). This is useful, in cases when color-management-aware software does not document the behavior."

Here are some examples

Regression tests

Right now it is pretty clear lcms2 would need an exhaustive test bed if we want some sort of robustness in the code. I'm spending a lot of time with this program. It began as a small one and now it is a huge file of 5600 lines. Maybe I should split it in several modules...

My intent is to have a test on every single feature. Since this is close to impossible, the actual test is focused on usual cases. Now the question, what are usual cases? Sure, sRGB to screen and aRGB to printer output are pretty common, but there are people over there using lcms1 to do multi-ink separations on 12 channels. Why I should not check this case as well? Another big area that deserves careful testing are plug-ins. How should the code base react to a wrong plugin? is a segfault admissible in this particular case?