When Delphi switched to MSBuild for compiling the projects, one thing got lost at the command line: hints. The compiler still outputs them but the MSBuild task Borland.Build.Tasks.Delphi eats them and at the command line you only see warnings, errors and fatal errors. Hints are only represented by an empty line. If your project outputs many hints you will see a lot of empty lines scrolling the console window but not the hint text.

Embarcadero Delphi für Win32 Compiler-Version 22.0 Copyright (c) 1983,2010 Embarcadero Technologies, Inc. <<<<< here should be a hint, but it isn’t visible >>>>> 12 Zeilen, 0.00 Sekunden, 13508 Byte-Code, 13100 Byte-Daten. Der Buildvorgang war erfolgreich. 0 Warnung(en) 0 Fehler

With Delphi XE2 this bug was fixed, intentional or unintentional, I don’t know, but the Quality Central report #67433 is still open. But what about older Delphi versions? The policy with Delphi was always/is to stop releasing updates for older versions when a new releases comes out (Delphi 7 being the only exception as Delphi 8 was .NET only). This allows us to do some binary patching in order to fix bugs without fearing that future updates would fail to install or skip installing the update version of our modified files.

How the patch works

The Borland.Build.Tasks.Delphi assembly uses a case-sensitive Regular Expression to identify hints from the compilers console output. Because the RegEx is case-sensitive, changing the “Hint” to “hint” in the command line compiler dcc32 will give us the result we want. The MSBuild task can’t find hints anymore and outputs them as plain text like the compiler’s copyright message. And for the localized version, we also need to patch the localized word for “Hint” in the dcc32.DE/FR/JP/… file. I use the German localized version so the example will show how to do it with the dcc32.DE file.

Why not patching the Borland.Build.Tasks.Delphi.dll ? The assembly is signed and MSBuild (.NET) won’t load the modified file. The locale resource file Borland.Build.Tasks.Delphi.resources.dll is not signed and can be modified but why patch two different files if you can patch the dcc32’s locale resource file.

The IDE compiler is not affected by the patch, so “Hint” will still be “Hint” in the IDE. Only the command line compiler will output “hint” instead of “Hint”.

Patching the command line compiler

1. You need a HexEditor. I’m using HxD.
2. You need to backup the file $(BDS)\bin\dcc32.exe
3. If you are using a localized version of Delphi, you also need to backup the locale file dcc32.de, dcc32.fr, dcc32.jp, …

Patching dcc32.exe

1. Copy the dcc32.exe file to a directory where you have write access
2. Open dcc32.exe in your hex editor and search for the unicode text “Hint” (hex: 48 00 69 00 6E 00 74 00). The text appears two times. The one that is followed by the unicode text “Warning” is the one you are searching for. It should be the first match. The second match is the unicode text “hint message” that we don’t want to patch.
3. Replace the upper case “H” with a lower case “h”
4. Save the file and copy it back to the $(BDS)\bin directory. (Have you made a backup of the original file?)

Patching locale dcc32.DE

If you only have the English version of Delphi you can skip this step.

If you want the locale compiler output to stay the way it is because you don’t like lower case words, you can skip this step and instead patch the Regular Expression in the MSBuild task’s resource file Borland.Build.Tasks.Delphi.resources.dll. (in German a lower case “hinweis” just looks weird).

1. Copy the dcc32.DE file to a directory where you have write access
2. Open dcc32.DE in your hex editor and search for the unicode text “Hinweis” (engl. hint) (hex: 48 00 69 00 6E 00 77 00 65 00 69 00 73 00). The text appears two times. The one that is followed by the unicode text “Warnung” (engl. Warning) is the one you are searching for. It should be the first match. The second is the unicode text “Hinweismeldung” (engl. hint message) that we don’t want to patch.
3. Replace the upper case “H” with a lower case “h”.
4. Save the file and copy it back to the $(BDS)\bin directory. (Have you made a backup of the original file?)

Embarcadero Delphi für Win32 Compiler-Version 22.0 Copyright (c) 1983,2010 Embarcadero Technologies, Inc. test.dpr(5) hinweis: H2164 Variable 'I' wurde deklariert, aber in 'T' nicht verwendet 12 Zeilen, 0.00 Sekunden, 13508 Byte-Code, 13100 Byte-Daten. Der Buildvorgang war erfolgreich. 0 Warnung(en) 0 Fehler

My changes to the directory cache in version 4.6.5 that I released yesterday introduced a new bug where the debugger wasn’t able to find the unit source files if you debugged project groups with relative library paths.

I’m sorry for that bug. I had a test case for it from the 4.6.1 release but I was more into testing the compiler than starting the debugger with this test case.

Download:
Name IDE Version File Size Downloads Added IDE Fix Pack 2007 4.4 2007 IDEFixPack2007Reg44.zip 61.85 KB 2401 times 2011-08-28 IDE Fix Pack XE2 4.6.6 XE2+UP3 IDEFixPackXE2Reg466.zip 152.4 KB 1536 times 2012-01-04 IDE Fix Pack XE 4.6.6 XE IDEFixPackXEReg466.zip 103.11 KB 914 times 2012-01-04 IDE Fix Pack 2009 4.6.6 2009 IDEFixPack2009Reg466.zip 135.22 KB 297 times 2012-01-04 IDE Fix Pack 2010 4.6.6 2010 IDEFixPack2010Reg466.zip 118.73 KB 745 times 2012-01-04

Changelog from 4.6.5 to 4.6.6 (2012-01-04)

• Fixed: Last directory cache fix broke the debugger’s source file search if you use relative paths

With version 4.6.1 I fixed a directory cache bug but also introduced a new one. Units in the DPR’s directory weren’t found anymore and the existing DCU file was used instead of recompiling the unit. In my tests I hadn’t cleared the DCU output directory and so the compiler always took the DCU from a former test, so I didn’t run into this bug while testing the code.

The changes to the directory cache that I did, make it a little bit slower because the directory cache is now rebuilt when the current directory changes instead of disabling relative paths what it did before.

Another patch that I’ve added improves the performance of the Delphi Editor’s undo operation, so it can compete with the large unit “contest” on http://delphitools.info/2011/12/29/synedit-performance-bragging-rights.

Download:
Name IDE Version File Size Downloads Added IDE Fix Pack 2007 4.4 2007 IDEFixPack2007Reg44.zip 61.85 KB 2401 times 2011-08-28 IDE Fix Pack XE2 4.6.6 XE2+UP3 IDEFixPackXE2Reg466.zip 152.4 KB 1536 times 2012-01-04 IDE Fix Pack XE 4.6.6 XE IDEFixPackXEReg466.zip 103.11 KB 914 times 2012-01-04 IDE Fix Pack 2009 4.6.6 2009 IDEFixPack2009Reg466.zip 135.22 KB 297 times 2012-01-04 IDE Fix Pack 2010 4.6.6 2010 IDEFixPack2010Reg466.zip 118.73 KB 745 times 2012-01-04

Changelog from 4.6.1 to 4.6.5 (2012-01-02)

• Fixed: Directory cache didn’t find units in the DPR’s directory anymore
• Added: Editor undo for a large undo operation is a lot faster

IDE Fix Pack 4.6.1 fixes two bugs. The directory cache wasn’t able to find files if CodeInsight was invoke in the uses-clause. This is caused by the IDE that forgets to change the current directory to the editor file’s project directory when it calls into the compiler. For the original compiler code this wasn’t a problem because it not only looks for the files in the relative paths but also in the absolute paths that it got from resolving the relatives paths when it was initialized. My directory  cache on the other side finds the files with their relative paths in the cache and tells the compiler that it found the file, even if the current directory is wrong for the relative path.
To solve this bug I not only made the directory cache aware of SetCurrentDirectory calls but also fixed the IDE’s bug, which now sets the correct current directory before calling the compiler.

The other bug that this version fixes is an access violation that can occur when invoking CodeInsight after a successful compilation. The compiler seems to sometimes keep a reference to a memory block that it returned to the memory manager and that my “Release unused compiler memory” patch gave back to the OS.

Download:
Name IDE Version File Size Downloads Added IDE Fix Pack 2007 4.4 2007 IDEFixPack2007Reg44.zip 61.85 KB 2401 times 2011-08-28 IDE Fix Pack XE2 4.6.6 XE2+UP3 IDEFixPackXE2Reg466.zip 152.4 KB 1536 times 2012-01-04 IDE Fix Pack XE 4.6.6 XE IDEFixPackXEReg466.zip 103.11 KB 914 times 2012-01-04 IDE Fix Pack 2009 4.6.6 2009 IDEFixPack2009Reg466.zip 135.22 KB 297 times 2012-01-04 IDE Fix Pack 2010 4.6.6 2010 IDEFixPack2010Reg466.zip 118.73 KB 745 times 2012-01-04

Changelog from 4.6 to 4.6.1 (2011-12-27)

• Fixed: Directory cache didn’t find files if CodeInsight was invoked in a uses-clause
• Fixed: Possible access violation during unit clean up

The new IDE Fix Pack 4.6 mainly addresses performance and memory leaks in the compiler and debugger. This version also fixes a race condition from the XMLSerializer cache in 2009-XE that sometimes caused a dead lock during the IDE start.

Version 4.6 supports the just released XE2 Update 3. The reason for the incompatibility is that Embarcadero changed some records in the compiler (what is their right) and all my hard coded offsets didn’t work anymore. Usually my patching code should catch those changes and disable the failing patches. But due to the number of RAD Studio version that I support, I changed many byte sequences, that I use to find the functions in memory, and inserted wild card bytes where memory offsets are, so I can patch all compiler versions with the same byte sequence. Thankfully, Embarcadero was kind enough to add the “incompatibility problem” to the XE2 Update 3 release notes.

Starting with IDE Fix Pack 4.6 for XE2, I added code that checks for the IDE’s version number and it won’t allow you to start the IDE with an IDE Fix Pack version that wasn’t compiled with the same RAD Studio Update. So IDE Fix Pack 4.6 for XE2 can only be used with XE2 Update 3. For all previous or future RAD Studio updates, it will show you a dialog and then terminate the IDE.

Download:

Name IDE Version File Size Downloads Added IDE Fix Pack 2007 4.4 2007 IDEFixPack2007Reg44.zip 61.85 KB 2401 times 2011-08-28 IDE Fix Pack XE2 4.6.6 XE2+UP3 IDEFixPackXE2Reg466.zip 152.4 KB 1536 times 2012-01-04 IDE Fix Pack XE 4.6.6 XE IDEFixPackXEReg466.zip 103.11 KB 914 times 2012-01-04 IDE Fix Pack 2009 4.6.6 2009 IDEFixPack2009Reg466.zip 135.22 KB 297 times 2012-01-04 IDE Fix Pack 2010 4.6.6 2010 IDEFixPack2010Reg466.zip 118.73 KB 745 times 2012-01-04

Changelog from 4.5 to 4.6 (2011-12-15)

• Added: After compile/debug unused compiler memory is released
• Added: LiveBindings slowed down the IDE (XE2) [XE2 Update 3 fixed it differently]
• Added: Compiler Package optimization
• Added: Delphi-Debugger GetFileIndex optimization
• Added: Debugger memory leak fix (XE2)
• Added: Memory leak fix for Debugger’s GetModuleFileNameEx calls (2009-2010)
• Added: Improved the performance when debugging the IDE (2009-2010)
• Improved: C++ Debugger symbol reader performance
• Fixed: XML Serializer cache race condition (2009-XE)

The updated version AsyncCalls 2.99 introduces the new methods CancelInvocation and Forget to the IAsyncCall interface. The first allows you to cancel not yet started tasks whereas the later allows you to unlink the task from the IAsyncCall interface. With “Forget” there won’t be a thread synchronization if the interface reference goes out of scope, but you also loose the ability to use the IAsyncCall interface to access the “forgotten” task.

Download:
Name IDE Version File Size Downloads Added AsyncCalls 2.98 5-XE2 AsyncCalls298.zip 33.98 KB 379 times 2011-11-24 AsyncCalls 2.99 5-XE2 AsyncCalls299.zip 35.7 KB 885 times 2011-12-14

Changelog:

• Version: 2.99 (2011-12-14):
  • Added: IAsyncCall.CancelInvocation method
  • Added: IAsyncCall.Forget method

I have uploaded a new build of the JCL and JVCL Binary Installers for XE2. The installers are based on JCL svn revision 3634 and JVCL svn revision 13184. They are binary installers which means that they do not need the dcc32.exe/dcc64.exe to install like the source code installers that are usually used for the JCL and JVCL. The binary installers can be used to install the JCL and JVCL if you have the RAD Studio XE2 Starter Edition or you want to evaluate XE2 with the Evaluation version.

JCL Binary Installer for XE2
JVCL Binary Installer for XE2 (requires JCL Binary Installer)

The developer snapshot 3 fixes the BCB 2009 issues and makes the debugger slightly faster than it already was with dev 2. I still have a lot of work to do in order to eliminate most of the remaining ReadFile calls.

BETA download table:
Name IDE Version File Size Downloads Added DelphiSpeedUp 3.1-untested RC for Delphi 6 6 DelphiSpeedUpV31D6.7z 74.59 KB 404 times 2011-04-22 DelphiSpeedUp 3.1-untested RC for Delphi 2006 2006 DelphiSpeedUpV31D2006.7z 106.03 KB 478 times 2011-04-22 DDevExtensions 2.4.2 7/2007 speed 7, 2007 DDevExtensions242Setup7_2007.zip 535.43 KB 983 times 2011-08-02 fastdcc 4.5 for 2009 2009 fastdcc2009v45.7z 87.55 KB 100 times 2011-11-02 fastdcc 4.5 for XE XE fastdccXEv45.7z 120.7 KB 241 times 2011-11-02 fastdcc 4.5 for XE2 XE2 UP2 only fastdccXE2v45.7z 129.52 KB 314 times 2011-11-02 fastdcc 4.6 for XE2 XE2 UP3 only fastdccXE2v46.7z 130.29 KB 144 times 2011-12-15

With IDE Fix Pack 4.6 dev 2 I’ve added a performance boost to the debugger for C++Builder projects. In a simple test project, a function in the comp32x.dll is called 89,428 times and it takes more than 7 seconds to answer those calls. So this function is very slow. I’ve now added a lookup hash table that reduced the number of calls to the original function to 5,510 with 0.7 seconds. That should make stepping through C++ code a lot faster.

BETA download table:
Name IDE Version File Size Downloads Added DelphiSpeedUp 3.1-untested RC for Delphi 6 6 DelphiSpeedUpV31D6.7z 74.59 KB 404 times 2011-04-22 DelphiSpeedUp 3.1-untested RC for Delphi 2006 2006 DelphiSpeedUpV31D2006.7z 106.03 KB 478 times 2011-04-22 DDevExtensions 2.4.2 7/2007 speed 7, 2007 DDevExtensions242Setup7_2007.zip 535.43 KB 983 times 2011-08-02 fastdcc 4.5 for 2009 2009 fastdcc2009v45.7z 87.55 KB 100 times 2011-11-02 fastdcc 4.5 for XE XE fastdccXEv45.7z 120.7 KB 241 times 2011-11-02 fastdcc 4.5 for XE2 XE2 UP2 only fastdccXE2v45.7z 129.52 KB 314 times 2011-11-02 fastdcc 4.6 for XE2 XE2 UP3 only fastdccXE2v46.7z 130.29 KB 144 times 2011-12-15

I have uploaded a development snapshot of IDE Fix Pack 4.6 dev 1 that addresses the XE2 LiveBindings IDE performance problem. Let’s see if this optimization is enough or if I need to add more complex optimizations.

BETA download table:
Name IDE Version File Size Downloads Added DelphiSpeedUp 3.1-untested RC for Delphi 6 6 DelphiSpeedUpV31D6.7z 74.59 KB 404 times 2011-04-22 DelphiSpeedUp 3.1-untested RC for Delphi 2006 2006 DelphiSpeedUpV31D2006.7z 106.03 KB 478 times 2011-04-22 DDevExtensions 2.4.2 7/2007 speed 7, 2007 DDevExtensions242Setup7_2007.zip 535.43 KB 983 times 2011-08-02 fastdcc 4.5 for 2009 2009 fastdcc2009v45.7z 87.55 KB 100 times 2011-11-02 fastdcc 4.5 for XE XE fastdccXEv45.7z 120.7 KB 241 times 2011-11-02 fastdcc 4.5 for XE2 XE2 UP2 only fastdccXE2v45.7z 129.52 KB 314 times 2011-11-02 fastdcc 4.6 for XE2 XE2 UP3 only fastdccXE2v46.7z 130.29 KB 144 times 2011-12-15

The design-time packages of the new LiveBindings that were introduced with XE2 are like sand for the IDE’s performance. The more open files you have the slower it becomes to invoke CodeInsight and to switch from Unit to Design and back. Disabling those packages gives you the performance back that you were used to in XE.

The Delphi 2010 Linker has a memory leak that becomes larger and larger the often you compile projects. And with my change in the size of the linker’s buffer from 32 KB to 5 MB I made this memory leak worse. Delphi 2009 doesn’t have that leak and it was fixed with XE. The 4.51 release is for Delphi 2010 only as the memory leak only affects 2010.

Change log from 4.5 to 4.51 (2011-11-11)

• Added: Fix for linker’s memory leak (2010 only)

Download:
Name IDE Version File Size Downloads Added IDE Fix Pack 2007 4.4 2007 IDEFixPack2007Reg44.zip 61.85 KB 2039 times 2011-08-28 IDE Fix Pack XE2 4.6.5 XE2+UP3 IDEFixPackXE2Reg465.zip 152.22 KB 103 times 2012-01-03 IDE Fix Pack XE 4.6.5 XE IDEFixPackXEReg465.zip 103.05 KB 66 times 2012-01-03 IDE Fix Pack 2009 4.6.5 2009 IDEFixPack2009Reg465.zip 135.21 KB 21 times 2012-01-03 IDE Fix Pack 2010 4.6.5 2010 IDEFixPack2010Reg465.zip 118.69 KB 54 times 2012-01-03

Question: Why do empty XE2 VCL projects have an EXE size of 7 MB compared to XE where the same project has only 0.9 MB?

Answer: You are compiling the project with the Debug configuration. And XE2 changed the default options for the Debug configuration to include the Debug Information into the EXE file. You can turn that option off in the project options under “Linker”. Or you can move them to an external *.tds file (the option under “Debug Information”). The debugger doesn’t need those Debug information because it gets the required information directly from the DCU and DCP files as it did in XE and older versions. I don’t know the reason why Embarcadero turned the option to include the Debug Information into the EXE on. Maybe the OSX debugger needs it (I don’t use that one). Or they wanted to allow us to debug the EXE in the IDE where the DCU und DCP files aren’t accessible (TDS information isn’t that good as the DCU’s debug information)

But even with that change the EXE file is 1.5 MB. So where do the other 600 KB come from? Almost 300 KB come from the new main icon resource that contains the icon in all Windows 7 required sizes. Another 200 KB come from the new Vcl.Themes unit that is hard linked from the Controls unit, so you can’t avoid it. The remaining 100 KB can be seen as the usual annual increase by changes in the RTL and VCL.

The IDE Fix Pack 4.5 now supports RAD Studio 2009, 2010, XE and XE2 with Update 2. With this release the RAD Studio 2007 support and earlier is discontinued. There won’t be new releases of DelphiSpeedUp/IDE Fix Pack for those IDE versions.

Change log from 4.4 to 4.5 (2011-11-02)

• Added: Improved Compile Progress dialog responsiveness and update interval.
• Added: Disables FileSystem cache to prevent double caching (directory cache) (XE2)
• Added: Disables GetUnitOf cache (XE2) and replaces the function by a much faster algorithm
• Added: Disables UnitFindByAlias cache as IDE Fix Pack already has a unitname hash (XE2)
• Added: Optimized compiler string functions
• Added: Delphi Linker write buffer size increased to 5 MB (original 32KB)
• Fixed: Some access violations due to the unitname hash
• Fixed: Directory cache caused problems with the debugger not finding the source files

Download:
Name IDE Version File Size Downloads Added IDE Fix Pack 2007 4.4 2007 IDEFixPack2007Reg44.zip 61.85 KB 1742 times 2011-08-28 IDE Fix Pack 2009 4.6 2009 IDEFixPack2009Reg46.zip 134.17 KB 28 times 2011-12-15 IDE Fix Pack 2010 4.6 2010 IDEFixPack2010Reg46.zip 118.01 KB 73 times 2011-12-15 IDE Fix Pack XE 4.6 XE IDEFixPackXEReg46.zip 102.12 KB 91 times 2011-12-15 IDE Fix Pack XE2 4.6 XE2 UP3 only IDEFixPackXE2Reg46.zip 150.21 KB 193 times 2011-12-15

And another developer snapshot. This time only one bug was fixed that caused {$L …} to not work as it was used to work and may have had other side effects in the compiler.

Name IDE Version File Size Downloads Added DelphiSpeedUp 3.1-untested RC for Delphi 6 6 DelphiSpeedUpV31D6.7z 74.59 KB 388 times 2011-04-22 DelphiSpeedUp 3.1-untested RC for Delphi 2006 2006 DelphiSpeedUpV31D2006.7z 106.03 KB 456 times 2011-04-22 DDevExtensions 2.4.2 7/2007 speed 7, 2007 DDevExtensions242Setup7_2007.zip 535.43 KB 879 times 2011-08-02 fastdcc 4.5 for 2009 2009 fastdcc2009v45.7z 87.55 KB 66 times 2011-11-02 fastdcc 4.51 for 2010 2010 fastdcc2010v451.7z 118.36 KB 122 times 2011-11-11 fastdcc 4.5 for XE XE fastdccXEv45.7z 120.7 KB 170 times 2011-11-02 fastdcc 4.5 for XE2 XE2 fastdccXE2v45.7z 129.52 KB 259 times 2011-11-02 IDE Fix Pack 2009 4.6 dev 3 2009 IDEFixPack2009Reg46dev3.zip 124.86 KB 60 times 2011-11-24 IDE Fix Pack 2010 4.6 dev 3 2010 IDEFixPack2010Reg46dev3.zip 107.87 KB 123 times 2011-11-24 IDE Fix Pack XE 4.6 dev 3 XE IDEFixPackXEReg46dev3.zip 93.66 KB 192 times 2011-11-24 IDE Fix Pack XE2 4.6 dev 3 XE2 IDEFixPackXE2Reg46dev3.zip 132.34 KB 297 times 2011-11-24

And another developer snapshot of IDE Fix Pack. This time the split in IDE Fix Pack and Compiler Speed Pack has begun in the XE2 version. This allows me to address the 32 bit and the 64 bit compiler.

What changed:

• The installer doesn’t cause Windows 7 to ask you if it installed correctly.
• XE2: IDE Fix Pack is split into IDE Fix Pack (the IDE part) and Compiler Speed Pack (the Compiler part).
• XE2: The 64 bit compiler is now patched by the Compiler Speed Pack for 64 bit. (Looks like the compiler comes from the exact same source code that the 32 bit compiler comes from. At least the part that I patch).
• GetUnitOf patch got another performance boost.
• The Compiler’s string functions are now patched with much faster code.

Name IDE Version File Size Downloads Added DelphiSpeedUp 3.1-untested RC for Delphi 6 6 DelphiSpeedUpV31D6.7z 74.59 KB 371 times 2011-04-22 DelphiSpeedUp 3.1-untested RC for Delphi 2006 2006 DelphiSpeedUpV31D2006.7z 106.03 KB 438 times 2011-04-22 DDevExtensions 2.4.2 7/2007 speed 7, 2007 DDevExtensions242Setup7_2007.zip 535.43 KB 810 times 2011-08-02 fastdcc 4.5 for 2009 2009 fastdcc2009v45.7z 87.55 KB 49 times 2011-11-02 fastdcc 4.51 for 2010 2010 fastdcc2010v451.7z 118.36 KB 77 times 2011-11-11 fastdcc 4.5 for XE XE fastdccXEv45.7z 120.7 KB 118 times 2011-11-02 fastdcc 4.5 for XE2 XE2 fastdccXE2v45.7z 129.52 KB 184 times 2011-11-02 IDE Fix Pack 2009 4.6 dev 3 2009 IDEFixPack2009Reg46dev3.zip 124.86 KB 39 times 2011-11-24 IDE Fix Pack 2010 4.6 dev 3 2010 IDEFixPack2010Reg46dev3.zip 107.87 KB 52 times 2011-11-24 IDE Fix Pack XE 4.6 dev 3 XE IDEFixPackXEReg46dev3.zip 93.66 KB 83 times 2011-11-24 IDE Fix Pack XE2 4.6 dev 3 XE2 IDEFixPackXE2Reg46dev3.zip 132.34 KB 119 times 2011-11-24

With the help of  an IDE Fix Pack user I was able to find a bug in the directory cache that caused the background compiler and the ErrorInsight parser to fail. The “forground” compiler and the CodeInsight compiler weren’t affected by this bug because they run in the main thread and not in a background thread.

The new developer snapshot also puts the directories of the resource files (*.res, *.dfm) into the directory cache.

Name IDE Version File Size Downloads Added DelphiSpeedUp 3.1-untested RC for Delphi 6 6 DelphiSpeedUpV31D6.7z 74.59 KB 362 times 2011-04-22 DelphiSpeedUp 3.1-untested RC for Delphi 2006 2006 DelphiSpeedUpV31D2006.7z 106.03 KB 436 times 2011-04-22 DDevExtensions 2.4.2 7/2007 speed 7, 2007 DDevExtensions242Setup7_2007.zip 535.43 KB 800 times 2011-08-02 fastdcc 4.5 for 2009 2009 fastdcc2009v45.7z 87.55 KB 46 times 2011-11-02 fastdcc 4.51 for 2010 2010 fastdcc2010v451.7z 118.36 KB 69 times 2011-11-11 fastdcc 4.5 for XE XE fastdccXEv45.7z 120.7 KB 107 times 2011-11-02 fastdcc 4.5 for XE2 XE2 fastdccXE2v45.7z 129.52 KB 167 times 2011-11-02 IDE Fix Pack 2009 4.6 dev 3 2009 IDEFixPack2009Reg46dev3.zip 124.86 KB 26 times 2011-11-24 IDE Fix Pack 2010 4.6 dev 3 2010 IDEFixPack2010Reg46dev3.zip 107.87 KB 39 times 2011-11-24 IDE Fix Pack XE 4.6 dev 3 XE IDEFixPackXEReg46dev3.zip 93.66 KB 64 times 2011-11-24 IDE Fix Pack XE2 4.6 dev 3 XE2 IDEFixPackXE2Reg46dev3.zip 132.34 KB 82 times 2011-11-24

I’ve put IDE Fix Pack 4.5 back into development as it wasn’t as stable as I wanted it to be. But this also allows me to put new optimizations into the code and improve the performance of older optimizations. Furthermore I added support for RAD Studio XE2+Update 1.

What has changed and what is new

First I made the unit name hash more stable. It sometimes doesn’t get rebuild after a unit is unloaded as the compiler seems to have multiple ways of doing the unload (or I haven’t found the right place to hook). So it looked for access violations when accessing the hash table to detect if a unit was removed from memory. Now it is notified by a hook in the compiler’s memory manager about releasing the memory of a unit.

The directory cache got some tweaks and sanity checks can be enabled by setting the environment variable IDEFixPack.ValidateDirCache=1. It allows you (or me) to identify bugs in the cache algorithm. But it will drastically slow down the compilation speed, especially in XE2 with its unit scope lookups.

The Compile Progress dialog slows down the compiler because both run in the same thread and GUI updates are slow. With this IDE Fix Pack the update time of the dialog is reduced to 80 milliseconds or a file name change. This allows to compiler to work on the code instead of waiting for the GUI to update. And then there is this optimization in the IDE’s code that only updates the dialog if the file name or the current line has changed. This would be a good optimization if there wouldn’t be RAD Studio XE and XE2 which load all units into memory when you “make” the project. And during this unit loading the compiler calls the progress dialog with the project’s file name and the last line in the project. So it is called thousands of time but no CPU time is spent on calling ProcessMessages due to this optimization what makes the dialog hang and you can’t cancel the compilation during that phase. I have changed that so you can press the Cancel button. The linker phase is still unresponsive as the compiler doesn’t call into the IDE during the linking phase.

Embarcadero (finally) introduced caches into the compiler. They have the UnitFindByAlias, FileSystem, SearchUnitNameInNS and GetUnitOf caches (http://blog.barrkel.com/2011/10/delphi-xe2-compiler-performance.html). IDE Fix Pack already had the UnitFindByAlias cache (=> unit name hash), FileSystem cache (=> directory cache) and SearchUnitNameInNS (=> part of the directory cache). What IDE Fix Pack didn’t have was the GetUnitOf cache. So I hooked into that function and profiled it with and without the cache. With my largest test project (4M+ LOC) the GetUnitOf function was called 52,625 times and took 16.36 seconds. After disabling the cache it needed 16.24 seconds. The problem with the cache is that the function is called in only 2% with the same parameter. But even without the cache 16 seconds is too slow for 52,625 calls. So I looked for a different algorithm and found one that utilizes a binary search instead of multiple linear searches. The result is that it now takes less than 300 milliseconds. “Unfortunately” GetUnitOf seems to be called not that often in normal units. Only my auto-generated unit (8 MB, 130,000 lines) and the MSOffice TLB import units caused the the compiler to call it that often.

The FileSystem cache is another cache that Embarcadero introduced. But it is no match for my directory cache. Especially not because it caches the same directory multiple times if it is accessed from different relative paths. My directory cache on the other hand uses the fastest way to get the directory listings (FindFirstFileEx under Windows 7+ with FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH). It also “pre-calculates” where DCU/DCP and PAS files are so it doesn’t have to ask every directory listing hash table, It gets the full and relative file name with one hash table lookup. My cache also keeps a list of “not found” files what helps to speed up the search for include files. So I have disabled the compiler’s FileSystem cache.

I don’t have much to say about the compiler’s UnitFindByAlias cache except that I have also disabled it. But not because it would be slow. The reason is that my other compiler optimizations depend on that information so I use my unit name hash instead of the compiler’s cache.

The SearchUnitNameInNS cache is the only cache that I have left active because my implementation doesn’t work. It isn’t cleared at the right time and returns outdated information. For example: The compiler calls the function with “Classes” and on the first call it returns “Classes”. So my patch puts “Classes”=>”Classes” into its hash table. But when the function is called the second time with “Classes” the original function returns “System.Classes” whereas my cache would return “Classes” from the hash table lookup.

Name IDE Version File Size Downloads Added DelphiSpeedUp 3.1-untested RC for Delphi 6 6 DelphiSpeedUpV31D6.7z 74.59 KB 353 times 2011-04-22 DelphiSpeedUp 3.1-untested RC for Delphi 2006 2006 DelphiSpeedUpV31D2006.7z 106.03 KB 431 times 2011-04-22 DDevExtensions 2.4.2 7/2007 speed 7, 2007 DDevExtensions242Setup7_2007.zip 535.43 KB 777 times 2011-08-02 fastdcc 4.5 for 2009 2009 fastdcc2009v45.7z 87.55 KB 34 times 2011-11-02 fastdcc 4.51 for 2010 2010 fastdcc2010v451.7z 118.36 KB 49 times 2011-11-11 fastdcc 4.5 for XE XE fastdccXEv45.7z 120.7 KB 87 times 2011-11-02 fastdcc 4.5 for XE2 XE2 fastdccXE2v45.7z 129.52 KB 130 times 2011-11-02 IDE Fix Pack 2009 4.6 dev 2 2009 IDEFixPack2009Reg46dev2.zip 120.34 KB 2 times 2011-11-22 IDE Fix Pack 2010 4.6 dev 2 2010 IDEFixPack2010Reg46dev2.zip 104.26 KB 5 times 2011-11-22 IDE Fix Pack XE 4.6 dev 2 XE IDEFixPackXEReg46dev2.zip 89.6 KB 7 times 2011-11-22 IDE Fix Pack XE2 4.6 dev 2 XE2 IDEFixPackXE2Reg46dev2.zip 128.05 KB 15 times 2011-11-22 fastdcc

All the above changes are also in the fastdcc.exe command line compiler patcher.

There are 2 ways to use fastdcc.exe.

1. Using fastdcc.exe directly

1. Extract the 7z file into your $(BDS)\bin directory.
2. start fastdcc.exe as if it was dcc32.exe.

1. Replacing dcc32.exe by fastdcc.exe

1. Extract the 7z file into your $(BDS)\bin directory.
2. Rename dcc32.exe, dcc32.jdbg and dcc32.de/fr/jp to dcc32compiler.exe/jdbg/de/fr/jp.
3. Rename fastdcc.exe to dcc32.exe
4. Rename fastdccHook.dll to dcc32Hook.dll
5. start dcc32.exe or msbuild.

  Why are there no compiler optimizations for the 64 bit and MacOS compiler?

I haven’t had a look at them (from a “patch point of view”). And I don’t know if I will invest time in them in the XE2 time frame as it would mean that I have to maintain 6 compilers and 4 IDEs. Guess why I have discontinued Delphi 7/2007 when XE2 was released. Delphi 2009 support will be there for a long time as I actively use it at work.

C++ Builder XE/XE2 Patch

I almost forgot that IDE Fix Pack for XE and XE2 contain another experimental patch for C++Builder XE/XE2. I have replaced the “String hash table” that the C++ compiler uses for all strings. In XE if you create a new C++ VCL Forms Application and press Ctrl+Space in the editor, the C++ compiler calls the “findStrEntry” (what I called it) 521,001 times with 76,641 unique strings. The original implementation uses a very bad hash function. It has a hash table with max. 32,768 buckets but findStrEntry puts the 76,641 unique strings in only 4,416 buckets. The largest number of strings in a bucket is 74. And 2,295 buckets have more than 9 strings in the linear list. Only 724 strings have a bucket for themselves.
I have rewritten the findStrEntry function with a much better hash function. And it now uses 28,750 buckets. The largest number of strings a bucket is 20. Only 302 strings have a linear list with more than 9 strings and 7,928 strings have a bucket for themselves.

Let’s sum it up

findStrEntry calls 521,001 unique strings 76,641 max. buckets 32,768 original: buckets used 4,416 original: max. number of strings in a bucket 74 original: buckets with more than 9 strings 2,295 original: buckets with 1 string 724 new: buckets used 28,750 new: max. number of strings in a bucket 20 new: buckets with more than 9 strings 302 new: buckets with 1 string 7,928