Open Watcom Reflections

Open Watcom 2.0?

2011-10-06T13:14:00.000-04:00

I see that it has been almost exactly one year since I last posted to this blog. That is far too long to go without some sort of update. So what is happening in the world of Open Watcom? There actually isn't much to report. The project has been almost as quiet as my blog. There have been a number of minor improvements added to the code base, however, and thus I'm planning to organize a release of Open Watcom 2.0 in a few months. I want to get the changes into the general distribution. I also, quite frankly, want to let the world know that Open Watcom is still alive.

Code Generation for x86-64

2010-10-05T20:46:00.004-04:00

Currently Open Watcom officially provides code generators for two architectures: 16-bit and 32-bit x86 systems. Although the 16-bit x86 architecture is old, there are people still using it. Open Watcom is one of the few actively developed compiler suites that supports it. We consider this a feature.

However, the world is moving forward and many systems in use today are now 64-bit systems using the x86-64 architecture. These systems can normally run 32-bit x86 programs without any problems so such programs generated by Open Watcom continue to work on them. In fact, the Open Watcom tools themselves, all compiled with Open Watcom, run fine on my 64-bit Windows 7 system. However, many people have asked what plans we have for supporting true 64-bit code generation.

The short answer is, "We don't have any plans." It's not because we think it's a bad idea to support x86-64. On the contrary adding such support would be a wonderful addition to Open Watcom. The problem is a lack of developer resources. Adding support for a new architecture is far from trivial. It requires a new (or updated) code generator, of course, but also support in the debugger, the assembler, and various other tools. Right now the active contributors to Open Watcom all have other projects that consume their limited available time.

Interestingly the Open Watcom project does have a 64-bit code generator for the Alpha architecture. In fact a fairly functional set of Open Watcom tools can be compiled for the Alpha version of Windows NT. At least that was true at one point in time. The Alpha support was never officially released, but it shows that there is a history of 64-bit support, in some form, in Open Watcom.

One interesting approach for adding x86-64 support to Open Watcom might be to connect the Open Watcom front end to LLVM. I blogged about this last year. Doing that would be a major piece of work as well, but it might have much greater long term pay off than adding support for just a single new target. Of course it's all just talk until someone comes along who takes concrete steps toward implementing something.

What's in a Version Number?

2010-08-13T06:47:00.003-04:00

The current official release of Open Watcom is version 1.9. This leads one to wonder if the next release will be version 1.10 or version 2.0. The choice is important since many people assume that a 2.x version will be significantly different than previous 1.x releases.

In fact, the step to the next release of Open Watcom will probably not be much greater than the step between any of our other releases. There will be enhancements and bug fixes, of course, but not significantly more than between version 1.8 and 1.9 (for example). This suggests that the new release should be version 1.10 and that we should save the 2.0 moniker for some future and more special version.

However, I have a general dislike for version numbers like 1.10... particularly given that we have been using only two digit version numbers so far. For one thing it is ambiguous. Is that version "one point ten" or version "one point one, subversion zero?" Also when file names are sorted 1.10 appears after 1.1 and before 1.2, suggesting the wrong interpretation of the ambiguity. It would be different if we had been using 1.0x for the releases all along.

Thus it is my intention to number the next release of Open Watcom as 2.0 despite the fact that there will be nothing "special" about the release. It will just be another step in the sequence from 1.8 through 1.9. Will we be misleading the public who might expect something significant from a 2.0 release? I don't think so. Compared to Open Watcom 1.0, the 2.0 release will be a significant enhancement. Someone who has not been following the project might be tempted to take another look, and that's entirely reasonable. Open Watcom 2.0 will be a big step from Open Watcom 1.0 and it would be good to attract the attention of those who have not looked at Open Watcom since the early days.

On a more technical note, the value of the __WATCOM__ macro in Open Watcom 2.0 will be 2000. So far all the Open Watcom 1.x releases have used a value of 12x0, reflective of the fact that Open Watcom is essentially Watcom v12. Since the last commercial release of the Watcom compiler suite was version 11, this is entirely reasonable. However, setting __WATCOM__ in Open Watcom 2.0 to 1300 seems rather counter intuitive. Thus we are taking this opportunity to jump to a more direct numbering scheme for the __WATCOM__ macro. We don't anticipate this causing any problems since the value is still monotonically increasing and thus will work properly in relational tests.

Open Watcom 1.9 Released!

2010-06-02T14:56:00.005-04:00

Today I finalized the 1.9 release of Open Watcom. It is equivalent to Release Candidate 6 (I haven't been good about keeping this blog updated regarding release candidates). Some of the new features include support for RDOS, support for Windows 7 development in the Win32 API, IDEAL mode support in WASM, our own documentation compiler for OS/2, and the usual collection of fixes and minor enhancements. The release notes contain more details.

This has been another long release cycle... longer than expected or desired. We've been averaging over a year between releases but I would prefer a six to eight month pace instead. Open source development is supposed to be agile, but so far the Open Watcom project has not had that characteristic. Users who want to only work with official releases currently have to wait a rather long time to pick up new features and fixes.

The next release will be, of course, version 2.0. That's a significant milestone and it is natural for people to wonder what we have planned for it. I think, however, I will post separately about that issue. Stay tuned!

Open Watcom 1.9, Release Candidate 4

2010-04-20T08:42:00.002-04:00

Release candidate 4 of Open Watcom 1.9 is now ready to download and test. We are converging on a full release and it is my hope that this release candidate will be the last one before 1.9 is final. This RC contains some fixes to the Win32 API support and a number of other minor fixes.

Open Watcom 1.9, Release Candidate 3

2010-04-06T08:26:00.004-04:00

I posted Open Watcom 1.9 release candidate 3 today. This release candidate contains quite a few corrections to the distributed documentation, particularly in connection with the new HTML help documents that we are now supporting. A number of other (mostly minor) fixes are also included. As always, please direct any questions or comments about this release candidate to either myself or to the Open Watcom contributor's news group.

Open Watcom 1.9, Release Candidate 2

2010-03-26T13:35:00.002-04:00

As I write this I am uploading the next release candidate of Open Watcom v1.9. This release candidate fixes about a dozen issues that were identified with RC1. Please direct any questions or comments about this release candidate to either myself or to the Open Watcom contributor's news group.

Open Watcom 1.9, Release Candidate 1

2010-03-09T08:37:00.002-05:00

I realize that it has been a rather long time since I've posted anything in this blog. I apologize for that. I have many active projects and it can be sometimes difficult for me to find the time for casual conversation. The good news is that the Open Watcom project has not been stagnant during the past months. Today, in fact, I posted Release Candidate 1 of the next version of Open Watcom. You can download the release candidate from the usual download page.

Rather than taking space here discussing all the changes in Open Watcom 1.9, I will just refer you to the release notes posted on the Open Watcom wiki. That should give you an idea of some of the exciting new features in the upcoming release.

OS/2 Documentation Generation

2009-04-15T08:29:00.004-04:00

Open Watcom provides development tools for several host environments. Using Open Watcom one can develop programs on DOS, Windows (even the old Windows v3.1), OS/2, and Linux. To do this effectively, however, one needs to be able to read the Open Watcom provided documentation on each of these platforms as well. That documentation contains information about the Open Watcom tools and run time libraries. It is an essential reference for the Open Watcom user.

Internally the documentation is in a format known as WGML, a text based markup language developed originally at Waterloo University. Through a series of steps the WGML source is transformed to a platform specific format that can be read by a platform specific documentation viewer. This process allows us to use the same documentation source for all platforms which is obviously highly desirable.

The final step of the documentation tool chain is a "help compiler" that outputs the necessary platform specific format. For Windows we are using the free help compiler created by Microsoft for this purpose (it is a separate download, we do not distribute this tool with the Open Watcom source). Similarly on OS/2 we have been using the IBM help compiler ipfc to produce the OS/2 version of the documentation. Unfortunately ipfc has been something of a thorn in our side. Recent versions of the tool have a bug that causes it to crash when processing certain Open Watcom documents. However, older versions of the tool are unable to process other of the Open Watcom documents. Thus building the OS/2 documentation required an elaborate dance.

Build as many documents as possible with the new version of ipfc until the tool crashes.
Manually build the troublesome document with the old version of ipfc.
Build the remaining documents with the new version of ipfc.

What's more during the most recent release cycle I noticed other problems with the OS/2 documentation build that seemed to be related to incompatibilities between our latest documents and IBM's ipfc. Finally, because ipfc is only available on OS/2 it is necessary to use OS/2 to build the OS/2 help files. This is somewhat awkward for me since I can't build the entire system on the Windows machines that I typically use.

However, early in the 1.9 development cycle Lawrence Haynes surprised the rest of us by contributing a clean room reimplementation of ipfc, called wipfc to the Open Watcom project. Wow! Now we have our own OS/2 help compiler and are in control of its source code. We can compile wpifc for all our platforms so now we can build OS/2 help on Windows, Linux, and even DOS (as well as on OS/2, of course). Furthermore we can address the issues that have plagued ipfc in the past to further simplify the build process and, hopefully, provide better documents in the long run. In particular, wipfc can compile all the Open Watcom documents without crashing. That's a step forward.

The wipfc tool is not yet 100% correct, however. Although it can compile the entire documentation set, the documents produced have some (mostly minor) problems. It is my hope that these final issues can be worked out before the v1.9 release. A more interesting question, perhaps, is this: should we include wipfc in future Open Watcom distributions? The OS/2 community might find it useful to have access to an open source version of the tool that is actively maintained as part of an open source project. For that matter one might wonder if we should distribute the entire WGML tool chain to enable third parties to use WGML for their own documentation. That is, perhaps, a good topic for a later post.

By the way, as far as I know we do not have a documentation compiler or viewer for Linux (unless the DOS tools happens to work when compiled as a Linux applications... probably not right now). This is one aspect of Open Watcom's Linux support that needs to be improved before the status of the Linux host becomes non-"experimental."

I should also mention that the Open Watcom build system can create PDF versions of the documentation using the third party Ghostscript tool. This is done on the build servers; fresh PDF documents can be downloaded from them and PDF documents for recently released versions of Open Watcom can be found on the main Open Watcom web site.

Open Watcom 1.8 Released!

2009-02-24T14:59:00.004-05:00

After an extended release candidate cycle, the next version of Open Watcom is finally ready! The process of producing v1.8 took much longer than I expected and probably longer than necessary. What have I learned from this process? Several things...

We should release more often so that the amount of change in each release is less. This will make digesting those changes a little easier and hopefully reduce the complications that arise during the release process. It also has the very significant benefit of getting fixes and new features into the hands of the user community more quickly.
The release process needs to be streamlined. Right now there are many labor intensive steps that are being done manually. Although some steps require a significant amount of human judgment and may be difficult to automate, others could certainly be handled by (or helped by) appropriate scripting. If the release process was more automated it would be easier to make releases more often.
The release branch should get regular exercise. In the past I've mostly ignored the release branch in our Perforce depot until it was time to start making release candidates. I think it may be helpful to instead periodically branch the trunk over to the release branch and verify that there are no unforeseen problems (for example with merging changes or with building/testing the potential release). I imagine that issues will be easier to fix if they are discovered early and well before an actual release needs to be made.
There should be better coordination between my building of the release and contributor activities. I need to communicate better with the contributor community to make sure that planned changes are either completely in place or have not yet started before officially branching the release for the first time. It probably makes sense to have a feature freeze for a few weeks before the release and shift development activity to testing and cleanup during that time. Because of the small number of Open Watcom contributors we have not bothered to do this in the past, but I think such an approach help in the long run.

In the interests of making faster releases, I've decided to try and get version 1.9 out the door in six months or so, regardless of how much or how little is done during that time. This suggests a feature freeze some time in July, release candidates some time in August, and a final release by (say) September 1. Of course this schedule is arbitrary and subject to change, but I think sticking to something approximately like it would be healthy.

Open Watcom 1.8 Release Candidate 4

2009-02-11T22:19:00.002-05:00

Open Watcom 1.8 Release Candidate 4 is ready for evaluation by the community. One of the new features in this upcoming release is the offering of installers for DOS and Linux systems (as well, of course, as the usual installers for Windows and OS/2 systems). The DOS and Linux installers have been classified as experimental until now. Consequently there have been quite a few quirks in them that needed fixing. This release candidate addresses several of those quirks. In addition, a few minor fixes to some header files have also been included.

Unless something significant accidentally got broken between RC3 and RC4, it is likely that RC4 will end up being the final release. We should know in a few days! The new release candidate can be downloaded from the usual places.

Open Watcom 1.8 Release Candidate 3

2009-01-23T12:08:00.002-05:00

Open Watcom 1.8 Release Candidate 3 is ready for evaluation by the community. There are still a few issues identified in the earlier release candidates that have not yet been (intentionally) addressed in this latest one. However, I wanted to get this release candidate into the hands of the community sooner rather than later so that the fixes that have been applied so far can be evaluated.

Although there is a good chance that there will be an RC4, I believe the system is converging to a final release. Most of the problems identified at this point are fairly obscure and shouldn't affect most users. The new release candidate can be downloaded from the usual places.

Open Watcom 1.8 Release Candidate 2

2009-01-05T12:59:00.003-05:00

Open Watcom 1.8 Release Candidate 2 is ready for evaluation by the community. This release candidate fixes almost all of the issues identified in RC1. Several issues related to the new Win32 API headers and libraries have been fixed. The new release candidate can be downloaded from the usual places.

Open Watcom + LLVM?

2009-01-03T22:28:00.003-05:00

The Open Watcom project has respectable code generators for 16 bit and 32 bit x86 architecture processors. The project also has some experimental code generators for certain other architectures (for example, Alpha and MIPS) but they have never been sufficiently completed to warrant distribution.

One could argue that the x86 is in decline and that the future of that architecture is bleak. For example, many new desktop and server machines are using x86_64 architecture so supporting 64 bit code generation for that architecture would seem like a priority. Of course there are many other architectures worth supporting as well... both large and small. Also who knows what the future of processor design will bring. Considering the limited resources of the Open Watcom project, it seems unlikely that Open Watcom could ever hope to support more than a tiny subset of potentially interesting targets.

One possible solution would be to support some kind of virtual target and then use the tools provided by that virtual target to do final code generation onto whatever real targets the virtual target supports. In other words, what is needed is something like LLVM.

I am no LLVM expert, but as I understand it LLVM defines an intermediate language with a level of abstraction that is similar to assembly language. This intermediate language can be expressed in textual form (just like assembly language) or in a binary "bitcode" form. The LLVM toolkit then provides tools to link bitcode modules together and to do code generation to one of several supported target architectures. There are several nice things about this from Open Watcom's point of view.

LLVM is a very active project and it is likely that new targets will be added to it regularly as time goes on.
LLVM can do some advanced "link time" optimizations on the bitcode files.
LLVM provides various back end tools that operate on bitcode files, such as a library manager, a profiler, an assembler, and a debugger (I believe).

To tap into LLVM, what needs to happen is for Open Watcom to grow a code generator that takes the internal intermediate language used by the Open Watcom compilers and translates it into LLVM's IR ("intermediate representation"). At that point the tool chain would switch over the LLVM's tools for linking, and final code generation, etc. This would allow the Open Watcom compilers (and front end tools like WMAKE, the IDE, etc) to take advantage of the rich collection of targets and advanced features provided by LLVM.

Although the basic idea sounds simple, making this a reality would require a significant amount of work. For one thing, to do it nicely it would be desirable to compile the LLVM back end tools with Open Watcom. Alas, the LLVM compilation system is very Unix-centric; getting it to build with Open Watcom tools would no doubt be a challenge. Also the LLVM documentation warns that LLVM's source code "stresses" C++ compilers. I suspect compiling LLVM might require applying further fixes to Open Watcom C++ (which would be a good thing for its own sake). Of course getting the Open Watcom C and C++ runtime libraries to compile for LLVM might also be an involved process.

Ultimately I'd love to see an LLVM target, perhaps with subtargets for each supported processor architecture, offered to users of the Open Watcom installer. Selecting the LLVM target would install LLVM back end tools and generally enable LLVM based development in an otherwise Open Watcom environment.

To explore these ideas further, and just to get to know LLVM a little better, I downloaded LLVM v2.4 and successfully compiled the back end tools on my OpenSUSE 10.2 machine. My thought was to study the system and learn my way around its assembly language. At that point I might be ready to think about building an OW->LLVM code generator. I could rely on the gcc-compiled LLVM back end tools at first. That would allow me to experiment with an OW/LLVM combination without having to immediately fuss with getting LLVM to compile with Open Watcom.

While I'm on the subject of virtual targets, I am lead to wonder if it makes sense to chart out a similar plan for targeting the intermediate language used by the CLR (aka .NET). Hmm...

Open Watcom 1.8 Release Candidate 1

2008-12-22T15:29:00.003-05:00

After a rather long delay, Open Watcom 1.8 Release Candidate 1 is finally ready for public evaluation. It can be downloaded from the usual locations. I am pleased to say that the issues I discussed in my previous entry, as well as some other issues that were discovered since then, have all been fixed in this release candidate.

The path to RC1 was not as smooth as I might have liked. It took a lot longer to produce than I expected. One problem was that I am prone to being a perfectionist and I hesitate to release something unless everything about it is just right. Of course perfection is not a realistic goal especially considering the size of the Open Watcom project and the resources the contributors have to spend on it. Setting aside those unrealistic ideals help me finally move things along.

However, I think the real issue was much more mundane. I simply attempted to branch 1.8 prematurely considering the magnitude of the changes in this release. I was just too anxious to get RC1 out the door. A longish list of problems (meaning regressions relative to 1.7) appeared shortly after I made my initial branch, necessitating a longish list of fixes. After a while it became clear that just branching the entire project again was the easiest and most reliable way to ensure that all the necessary fixes eventually got into 1.8. That seems like a sure sign that the initial branching was too early.

At this point 1.8 appears reasonably stable. While I'm sure new problems will be found with the release candidate (after all, that's what release candidates are for), I do at least feel that RC1 is a true release candidate in every sense of the word. It's not just a development snapshot. It's not just a beta. It's the real deal.

I encourage everyone in the Open Watcom user community to download 1.8RC1 and try it out. As I've said before, this is a significant release with big changes. It will need all the exercise it can get!

Open Watcom v1.8 Delayed

2008-11-16T06:46:00.003-05:00

A few weeks ago I branched what would hopefully become Open Watcom v1.8 into the release branch of our Perforce depot. This process takes a snapshot of the developing code base and officially freezes the code that will be the basis for a release.

Alas, shortly after branching 1.8 and before I could prepare a release candidate Michal Necasek noticed a couple of significant problems with the system. One of the problems related to the handling of DLL imports under Win32. It is not possible with the current 1.8 to define a "local" function that satisfies a DLL import. That is, you can't override a Win32 API function using a function defined in your own code. This was possible with Open Watcom 1.7. The problem now occurs because the new Win32 API declares itself in a fully proper way, exposing a previously latent bug.

Of course people rarely import local functions, but it turns out that some people do. For example JWasm, an unofficial branch of Open Watcom's assembler, does exactly that. Currently JWasm can't be built using 1.8. I regard that as a significant problem.

The other problem discovered a bit later by Michal is related to the compiler's handling of the _ialias.h header and its interaction with the precompiled header feature. The _ialias.h header contains a number of #pragma include_alias directives that map long header file names to the short names used in the Open Watcom distribution. This mapping is fundamentally necessary because Open Watcom supports installation on FAT16 file systems where names are limited to eight characters (with a three character extension). Thus a header such as algorithm has to be shortened to algorith. Yet despite that programs that #include <algorithm> must still work.

The problem was that the compilers (both C and C++) were emitting incorrect output for the #pragma include_alias directives in _ialias.h when doing preprocessing only. In addition the information used by the precompiled header mechanism was getting confused, causing unnecessary header recompilations.

For many users neither of these problems are show stoppers. Indeed most users probably won't notice them at all. Yet the problems are regressions. In one case the compilation of a friendly branch is broken. In the other case, since the C compiler now includes _ialias.h automatically, a prior bug is being thrust onto users that previously didn't have to deal with it.

The question for me becomes: What should I do about release candidate one (RC1)? Normally release candidates are... well... candidates for release. When they are posted there should be no known show stopping problems with them. Once a release candidate has been checked by the community it should become the release with no changes other than renaming the distributed file. Posting a "release candidate" with known issues seems to go against that philosophy.

On the other hand, it is common for Open Watcom releases to go through two or three release candidates. Holding back RC1 for a couple of obscure regressions just slows down the checking of other parts of the system. There are users who don't bother testing the development code but who do aggressively check the release candidates looking for problems (a practice that I very much appreciate). Their input would be valuable even before the bugs I mentioned above are fully addressed. Yet I hesitate to ask those users to expend time and effort testing a "release candidate" that we know won't be the final release.

For the reasons above I decided to delay Open Watcom v1.8 RC1 for a while. In the time since I branched 1.8 in the Perforce depot some of the problems I've described here have been fixed (and a few other minor bugs have been found and fixed as well). However, as I write this there is still some more work to do. I am anxious to get RC1 into the hands of the user community, but I also don't want to rush the contributors who are giving their own time for no pay to make Open Watcom happen. Unlike a commercial endeavor we don't have a schedule to keep, and the main goal of our effort is to have fun. Being pressured is not fun.

Still, I may post an RC1 soon regardless of the state of the problems described above. If I post it with a suitable warning, testers will at least understand that an RC2 will definitely be forthcoming. I also know that some users would appreciate seeing an RC1 now regardless.

w32api2

2008-10-19T21:47:00.005-04:00

The commercial version of Watcom shipped with the Microsoft SDK headers and libraries to support programming for Windows platforms. When Watcom became Open Watcom the Microsoft SDK could no longer be shipped with it for legal reasons. Instead the free MinGW Win32 API headers and libraries were used. Switching to MinGW wasn't entirely trivial; there was some work involved to migrate the tools to the new library so that Open Watcom could build itself. However, this all happened before I got involved with Open Watcom so I don't know much about it.

Alas, incorporating a third party library such as MinGW into the core distribution of Open Watcom creates some maintenance problems as well. For various reasons, the raw MinGW headers are not 100% compilable with Open Watcom. Thus the version of MinGW that has traditionally shipped with Open Watcom has had a few adjustments made to it. Also, although it has been freshened several times during Open Watcom's evolution, the version of MinGW that is part of Open Watcom may be slightly behind the latest official version.

Operating system APIs are fundamental to the target platform. They form the basis of the supported language run time libraries and, directly or indirectly, all other libraries compiled for that platform. Having such an important library outside of our direct control has been troubling. In contrast the OS/2 API library (and for that matter the Linux and DOS API libraries) used by Open Watcom are part of the Open Watcom project. This is in many respects a much more comfortable situation.

Over the last several months David Golub has been working on a new Win32 API library, affectionately called "w32api2" by the Open Watcom contributors. David has done an incredible amount of work on this project and has essentially built single handedly a replacement for the MinGW Win32 API library. As I write this, w32api2 is positioned to become the official Win32 API library for Open Watcom starting with the upcoming 1.8 release.

One might reasonably ask, "Why do all this work re-inventing the wheel? Couldn't David's time have been better spent working on something that has not yet been done?" First, because Open Watcom is an open source project, people will only work on what they are interested in doing. David is obviously passionate about Win32 and if he wants to contribute to the project in this way, who am I to dissuade him? When people work for free, their only reward is the satisfaction of a job well done... and any job well done is good enough for me!

Yet there are also several highly practical reasons for creating w32api2.

We can address bugs found more easily and more directly than when we have to go through a third party who may or may not be responsive to our needs.
We can take into account the strengths and weaknesses of our compiler, resulting in cleaner code in the library and, potentially, in applications as well.
We can implement features we feel are important without having to convince a third party with a different agenda. For example, w32api2 has much more extensive support for Vista application development and much better compatibility with the Microsoft SDK than MinGW.
We are now in control of the platform API library we are using for Windows as well as for OS/2, Linux, and DOS.

Of course a change of this nature entails risk as well. David has done a fantastic job of exercising w32api2. He has compiled all the relevant sample programs in the Microsoft SDK (that don't use Microsoft specific extensions), several major third party libraries (such as wxWidgets and others), and of course demonstrated that the new API library can compile Open Watcom's own source code... including the GUI Windows tools. Numerous bugs have been found and fixed during this work.

If you are a Windows programmer you should definitely check out the upcoming 1.8 release candidates and look for problems with the new API library as applied to your code. There is still time to fix bugs between a release candidate and the final release, but those bugs can only be fixed if we know about them!

Open Watcom 1.8 and C++

2008-09-25T12:47:00.003-04:00

Thanks to the work of Christof Meerwald, the C++ compiler in the upcoming 1.8 release will be a major step forward relative to 1.7. The Open Watcom C++ compiler was originally designed long before C++ was first standardized. The commercial Watcom system was taken off the market before C++98 was finished, and so the compiler didn't get the chance to evolve with the standard as other C++ compilers did.

Because of this history, the internal architecture of the compiler was not well suited to the demands of Standard C++. Certain features of the current standard would have been very difficult to properly implement, not to mention the new features in the upcoming standard. To address this problem Christof gutted the compiler, rewriting and refactoring significant parts of it. Although dramatic, this action was necessary before the compiler could progress much further.

A relatively complete list of the new features now supported by the C++ compiler can be found on Christof's page on the Open Watcom wiki. Some of the features listed there are relatively obscure, but some are very significant. For example

The compiler now supports explicit template arguments for function templates. For example, consider the template as follows
```
template< typename T >
T f(int x, int y)
{
  T result;

  result = x + y;
  return result;
}
```
Open Watcom 1.7 follows the pre-standard behavior of the language and calls this definition an error because the template parameter T does not appear in the function's parameter list. Thus template argument deduction can't work and the template can't be used. However, in Standard C++ the definition is acceptable because it is possible to call the template using explicit template arguments. For example
```
float answer = f<float>(1, 2);
```
Open Watcom 1.8 will support this feature. This is an important advance because certain aspects of the C++ standard library rely on such templates (the facets of IOStreams).
The compiler now supports lazy template instantiation. This means, among other things, that only the methods of a class template that are actually used are instantiated. This is important because if a template is instantiated with a type for which certain methods don't make sense, the operations used in those methods are not relevant. For example, the std::list template has a method for sorting the list. In Open Watcom 1.7 this template can not be instantiated for a type without a suitable operator< because the instantiation of that method would fail. However, Open Watcom 1.8 will allow such types to be used provided no attempt is made to actually call std::list<T>::sort.
The compiler now allows out-of-line definitions of member templates and nested classes. In Open Watcom 1.7 member templates and nested classes had to be defined in-line. This put practical restrictions on the complexity of such templates and classes. This limitation has been lifted.

The features above are significant because they enable certain important programming techniques. These techniques have become common in modern C++ and, until now, Open Watcom has not been able to support them.

In addition, there have been many fixes and enhancements to the handling of function overloading, template friends, and conversions. Numerous bugs, some long outstanding, have been fixed. With this change, Open Watcom's C++ compiler has taken a major leap forward and is now reasonably close to conforming to the C++ standard (but not yet... there still remains work to be done!).

My personal interest in the C++ side of Open Watcom is primarily in the area of the C++ library. Although library progress has been slow lately for various reasons, compiler limitations and bugs were causing significant problems for library development. However, with Open Watcom 1.8 many of those limitations and bugs are gone. With any luck this means the C++ library may take a similar leap forward in version 1.9.

Of course a change of this magnitude comes with risks as well. Much work has been done by Christof, Daniel Cletheroe, Michal Necasek, myself, and others to ensure that the new compiler doesn't suffer any undesirable regressions. Could there still be some regressions we haven't caught? Of course. I encourage anyone interested in Open Watcom C++ to take the time to review the release candidates when they become available. This is not going to be a "business as usual" release!

Introduction

2008-09-07T08:21:00.009-04:00

Hello! My name is Peter Chapin, and this is my blog about the Open Watcom project. I've been involved with Open Watcom since late 2003. I used the commercial Watcom C/C++ compilers in my classes at Vermont Technical College (where I teach) during the early to mid 1990s. When the project became open source and could thus be freely downloaded by my students, I hoped to make use of it again. However, at that time the C++ compiler (especially) was considerably behind the current standard. Thus I joined the project to help modernize the C++ tools.

My initial contribution was to add the so-called "cname" headers (such as <cstdio>, <cstdlib>, etc) so that the facilities of the C library could be accessed through namespace std as required by the standard. I was telling my students this was the proper way to use the C library from C++ programs and yet Open Watcom, at that time, didn't support it. I'm pleased to say that my implementation of the cname headers follows the standard rigorously. In particular, the following program fails to compile with Open Watcom v1.7a because it tries to access a C library function through the global name space.

#include <cstdio>

int main( )
{
  printf( "Hello, World!\n" );
  return 0;
}

Unfortunately, and for reasons that are unclear to me, very few compilers follow this aspect of the standard.

After working on the cname headers, I needed to find a way to add support for the Standard Template Library. I considered trying to include an existing third party library such as STLport (or some other). In the end I decided to work on a completely new implementation of the STL, now called OWSTL. I made this decision largely for two reasons.

I wanted to create a library tailored to Open Watcom... that could take advantage of Open Watcom's features and yet also work around its bugs, all without the distraction of supporting many other unrelated compilers.
I wanted to learn about the details of the STL and what better way to learn is there than trying to implement it?

For more information see the Open Watcom STL page on the Open Watcom wiki. The OWSTL project is still ongoing. Although the library has a ways to go, with the help of others (especially Daniel Cletheroe) it has reached a point where it can be used for certain applications.

Starting in early 2007, I took over the role of maintainer for the Open Watcom project after Michal Necasek stepped down from the position. This means that I'm the one responsible for putting together the releases. I also take care of certain administrative duties such as interacting with our contacts at Perforce (who is generously supporting the Open Watcom project by providing a free license to their source code management system and paying for the host on which the Open Watcom project runs).

In this blog I hope to talk about my impressions of all things Open Watcom. I intend to talk about what's happening with the project, where I think the program is going (or should be going), and even tidbits about how to use some of Open Watcom's features. I hope to post fairly regularly so stop back often to see what's happening in the world of Open Watcom!