Backwards compatibility

Little CMS 2 is almost a full rewrite of 1.x series, so there is no guarantee of backwards compatibility. Having said this, if your application doesn’t make use of advanced features, probably all what you need to do is to change the include file from lcms.h to lcms2.h and maybe to do some minor tweaks on your code. Profile opening and transform creation functions are kept the same, but there are some changes in the flags.  Little CMS 2 does offer more ways to access profiles, so it is certainly possible your code will get simplified.  The basic parts where Little CMS 2 differs from 1.x series are:

• ·    Transform flags
• ·    Error handling
• ·    Textual information retrieval
• ·    New non-ICC intents
• ·    Floating point modes
• ·    Pipelines

On internal advanced functions, the underlying implementation has changed significantly. You still can do all what lcms1 did, but in some cases by using a different approach. There are no longer gamma curves or matrix-shaper functions. Even the LUT functions are gone. All that has been superseded by:

• ·    Gamma functions -> Tone curves
• ·    Matrix Shaper, LUT -> Pipelines
• ·    LUT resampling -> Optimization engine

There is no one-to-one correspondence between old and new functions, but most old functionality can be implemented with new functions.

What is new from lcms 1.x

First obvious question is “why should I upgrade to Little CMS 2.0”. Here are some clues:

Little CMS 2.0 is a full v4 CMM, which can accept v2 profiles. Little CMS 1.xx was a v2 CMM which can deal with (some) V4 profiles. The difference is important, as 2.0 handling of PCS is different, definitively better and far more accurate.

• It does accept and understand floating point profiles (MPE) with DToBxx tags. (Yes, it works!) It has 32 bits precision. (lcms 1.xx was 16 bits)
• It handles float and double formats directly. MPE profiles are evaluated in floating point with no precision loss.
• It has plug-in architecture that allows you to change interpolation, add new proprietary tags, add new “smart CMM” intents, etc.
• Is faster. In some combinations, has a x 6 throughput boost.
• Some new algorithms, incomplete state of adaptation, Jan Morovic’s segment maxima gamut boundary descriptor, better K preservation…
• Historic issues, like faulty icc34.h, freeing profiles after creating transform, etc. All is solved.

Documentation strategy

Little CMS documentation is hold in three different papers. First one is the tutorial. Its goal is to introduce the engine and to guide you in its basic usage. It does not, however, give details on all available functionality. For that purpose, you can use the API reference, which gives information on all the constants, structures and functions in the engine. The third document is the plug-in documentation. It details how to extend the engine to fit your particular purposes. You need some experience in the core API to write plug-ins, therefore, the plug-in API reference is somehow more advanced that the remaining two.

Aside documentation, there are sample programs that you can explore. Those are located in the “utils” folder. Those programs are also handy in isolation. This is the list of utilities, each one is documented elsewere.

• TiffICC
•  JpegICC
•  TransICC
•  LinkICC
•  TiffDiff
• psicc

Documentation!

Back from vacation. Lots of fun and some time devoted to LittleCMS. As a result, we have now some incipient documentation. There is a tutorial you can read to explore the differences between LittleCMS 1 and 2. The plug-in API is also documented, but the documentation lacks some explanations and is mostly unfinished. The API reference is a work in progress...

Unbounded CMM

With the drop I'm posting right now (September 19) most of the utilities are now working. You have a tifficc applier, a transicc calculator and a linkicc devicelink generator. As you can see, some names have changed. This is because icclink was clashing with Graeme Gill's icclink from Argyll. This latter is an outstanding utility, and many users may want to have both installed, so I changed the name and get an extra consistency bonus (now all lcms utility names are "whatever-icc")
Now you have transicc (former icctrans) and therefore you can check one of the new features of lcms2. That's what is called "unbounded CMM mode"

What is that? Well, with lcms2 you can use floating point values. That means,  you are no longer limited to 0..255 on 8 bits or 0...65535 on 16 bits, but on some profiles you can get out of those bounds and the CMM still works.

Not all profiles does accept that. You need profiles that are implemented using enterly math expressions. Some very simple profiles works in such way, for example AdobeRGB. The sRGB profile does not work because it has curves implemented as tables, but the built-in sRGB does work as instead of tables it uses parametric curves. Also some advanced profiles for digital cameras using multiprocessing elements may work in unbounded mode. Right now it is hard to find any of those, as they are described in an addendum to the ICC spec.

Let's check this feature. Using AdobeRGB to the built-in  Lab in transicc:

C:\lcms-2.0\bin>transicc -i AdobeRGB1998.icc -o *Lab
LittleCMS ColorSpace conversion calculator - 4.0 [LittleCMS 2.00]
Enter values, 'q' to quit

R? 10
G? 10
B? 10
L*=0.7287 a*=0.0000 b*=-0.0000

Nothing special, but let's try this one

R? 300
G? 300
B? 300

L*=114.6770 a*=0.0006 b*=-0.0005

Do you see where I'm going? 300 is above 8 bits, and L*=114.6 is a highlight, so here you have an example of what can be accomplised with this mode.

New drop

Ok, so here is the september drop. That's functionality complete, that means all the functionality is already there and now it remains debugging and stabilizing. I am still in good shape for the schedule, so hopefully the first beta will be available on mid october.

New in this drop:

• error logging supersedes error plug-in, which is no longer available (it was a bad idead, indeed)
• black preservation intents are working
• TAC detection is working.
• All tags and tag types are correcly read/written
• Gray/RGB/CMYK/Lab/XYZ
• Info fns with localization, unicode
• Segment maxima gamut boundary descriptor

... And now back to the documentation, that's what is taken most of my time. First documentation draft will be available on next drop.

Black is black (II)

The second black thing has nothing to do with black point compensation, but is still black-fashioned. That are the black preserving intents. No, this does not belong to normal ICC workflow. ICC has tried to address such need but still there is nothing in the spec.

Let's see what the issue is. Suppose we work in press. Press are very tied to standards, US press uses SWOP and European folks are more toward FOGRA. Japanese people uses other standards like TOYO, for example. Each standard is very well detailed and presses are setup to faithfully emulate any of these standards.

Ok, let's imagine you got an image ad, looking like that. This is a very usual flier, now just imagine instead of getting it in PDF, you get it as a raster file. Say in a CMYK TIFF, ready for a SWOP press. And you want to print in on a FOGRA27!!

Maybe you see no issue over here, but take a look on this:

LittleCMS ColorSpace conversion calculator - 4.0 [LittleCMS 2.00]

Enter values, 'q' to quit
C (0..100)? 0
M (0..100)? 0
Y (0..100)? 0
K (0..100)? 50

C=48.21 M=38.71 Y=34.53 K=1.09

That means, if I convert from SWOP to FOGRA27, the ICC profiles totally mess up the K channel, so a portion of the picture that originally is using only black ink, after the conversion, gets Cyan, Magenta, Yellow and well, a little bit of black as well. Now please realize what happens on all the text in the Flier.


Ugly. I guess the vendor who pays the bill will not be very pleased, right?

So I've added two different modes to deal with that: Black-ink-only preservation and black-plane preservation. The first is simple and effective: do all the colorimetric transforms but keep only K (preserving L*) where the source image is only black. The second mode is fair more complex ans tries to preserve the WHOLE K plane. I'm still checking this latter. If you want to give it a try, please download the new snapshot and build the TIFFICC utility. It already implements those new intents.

Back in black

I've been lately working hard in two black thingies. Namely, black point compensation and black preserving intents. Black preserving intents are worth of a post, and still there are some subtle bugs, so let's go fot the black point compensation first. BPC has been implemented in lcms for years. It works. So if ain't broken don't fix it. Well, not really. It seemed to me lcms2 would be a good chance to implement Adobe's BPC... oh, that's a nice story.

BPC is a sort of "poor man's" gamut mapping. It basically adjust contrast of images in a way that darkest tone of source device gets mapped to darkest tone of destination device. If you have an image that is adjusted to be displayed on a monitor, and want to print it on a large format printer, you should realize printer can render black significantly darker that the screen. So BPC can do the adjustment for you. It only makes sense on relative colorimetric intent. Perceptual and saturation does have an implicit BPC .

Works like magic but it is quite simple: just a plain linear scaling in XYZ space.

What turns out the real problem: It is easy to implement BPC as long as you can figure out what the black point is. The black point tag doesn't help. There is such quantity of bogus profiles that ICC has deprecated the tag in a recent addendum. It is up to the poor CMM to figure out what the black point is.

And it is a hard task tough. Ok, you may argue, just use the relative colorimetric in the input direction, feed RGB=(0, 0, 0) and ... wait, what about CMYK? Ok, CMY=(255, 255, 255,255)... Hum.. CMYK devices are ink limited, that is, the maximum amout of ink that a given paper can accept is limited (else ink will spread all over) so this doesn't work either. What lcms do, is to use perceptual intent to convert Lab of zero to CMYK and then this CMYK to Lab using rel. colorimetric. This works fine on all but some broken profiles. Well, Adobe's paper target those profiles.

The funny part is the complexity of Adobe's algorith. The perceptual trick is used as a seed on certain profiles, then black axis of a round-trip on L* is examined. If the difference is greater than 4, then a least-squares fitting of the curve to a quadratic curve should be done. The black point is the vertex.

In fact it is more complex as I'm ommiting constants depending on the intent and other gory details. Such complex. After the 6th time you read the paper, it begins to make sense.

But unfortunately it doesn't work to me. And unfortunately, it doesn't work for the ICC reference implementation CMM. We both get the same bad guess of L* = 625 for a given profile. And guess what? I've tried Photoshop CS4, and the reported black point does exactly match with my humble, perceptual to rel.col. algorith, so I will keep this one for a while.

New drop available

August drop is here. Working features are CMYK, black point compensation, all profile I/O ...
Enjoy!

Sleep your bugs

It hurts. Its a sort of burning... when you figure out where is the bug, you experience an urgent necessity of fixing it, NOW! Big mistake.

It is not clear if was Knuth or Hoare who coined the phrase:

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

It is indeed very true. I would like to add a humble corollary:

premature bugfixing is lame too.

So, here is my advice: Sleep your bugs.

I am not talking about typos. You know, misspelling 'l' and '1' for example (nobody should use 'le0' as var name, '1e0' is still a valid number) or failing to include a range check (last time I forget to place an 'if' I got a Pwnie nomination)

I am talking about logic bugs. Something wrong with the algorithm or the true logic of the program. Adding inconsistent features is another example.

Sleep those kind of bugs. Take good note of them, and wait a day or two in fixing. Probably you will find a neat way to deal with the issue without changing those hundreds of lines. Maybe not, but even in this case the solution is worth of the wait.

In my case the bug was black point detection. lcms2 is now an unbounded CMM that operates on floats. So the black point detection code was using floating point to do the calculations. Yep, it seems a nice feature: far more precision, etc. On the other hand, inks on floating point are represented as %, so the effective range is 0..100%, that is how Photoshop and others do work and I thought it makes sense to keep the feature in lcms2 as well.

Put both features together and you have a nice logic bug: extra code is needed on BP detection to deal with different ranges.

If you happen to be a professional programmer, you surely realize that my advice of postpone bug fixing goes against the schedule, the program manager and the planner. Sure, but that's lcms2, my pet project which has unlimited resources on time. Would be a dream for a commercial project, but this project is not commercial and therefore is not subjected to schedule tyranny.

By the way, a new drop of lcms2 is in the way and will be available in a day or two.

Less is more

If you have devoted some time to review the new API, maybe you have discovered an odd thing: there are some functions missing. Ok, you can blame me for remove *that* function doing exactly what you need. Of course that may be my mistake. But please consider perhaps I have good reasons to do that.

Let's take one example:

cmsReadICCMatrixRGB2XYZ(LPMAT3 r, cmsHPROFILE hProfile);

This function no longer exists in lcms 2.0 API.

Well, many people were using this function to retrieve primaries of a profile. So, for example, if you want to know which are the AdobeRGB primaries, just call the function with the right profile and here you go.

Seems easy, and useful, but trust me, it is not. The real reason d'être of this function is somehow surprising. Not because it is handy but because is precise. Please consider this piece of pseudo-code:

cmsXYZTRIPLE Result;
hXYZ = cmsCreateXYZProfile()
xform = cmsCreateTransform(hProfile, TYPE_RGB_DBL, hXYZ, TYPE_XYZ_DBL, INTENT_RELATIVE_COLRIMETRIC, 0)
cmsDoTransform(xform, {{ 1, 0, 0}, {0, 1, 0}, {0,0,1}}, &Result, 3)

Do you follow it? I create a transform from the profile (in RGB) to XYZ. Then I convert max of R, and B to XYZ. I am obtaining the primaries! Despite it seems more complex, this method is much better because is guaranteed to work in *any* profile, not only on matrix-shaper ones.

So, what is the point of having the old function? Easy: lcms 1.x was precission-limited to 16 bits, so you cannot obtain primaries with enough precision with the method described above. But that does not apply with lcms2, where you have an outstanding 64-bit double precission. Less is more in this particular case!

Linking tags

Implementing the tag link feature has been easier than I originally thought, but there are some caveats.

lcms2 does support some new features on read/writting profiles. You can use cmsReadTag/cmsWriteTag to read/write lcms objects like LUTs, tone curves and so. You can also use cmsReadRawTag/cmsWriteRawTag to read or write whatever you want, but poor library does not do any checking or understanding on whats going on.

You can also link tags to items created by any of those two methods. So far so good.

But wait, you can also write plug-ins to add more "understood" objects in cmsReadTag/cmsWriteTag and also you can write plug-ins to add new types for those objects.

In addition to all that, there is a brand new structure added as an addedum to ICC spec 4.2, that is the MPE or multi profile elements. This is worth of several comments in this blog. Basically that may make photographers and precission yonkies very happy as it includes *true* floating point numbers, among other things. There is a plug-in type devoted to MPEs. More to come.

So, if you consider all all those acess methods should be consistent, we have a nice mess. The good news are it seems to work, the bad news, we need more testing. But overall I see progress.

Fear not!, maybe I will even accomplish the schedule and realease the whole thing on November.