Tuesday, January 17, 2012

Hack Intraweb and make it faster!

I'm always looking for a way to make my applications faster. During last couple of days I've been investigating an - already fast - Intraweb application.

1) Intraweb rendering implementation

Lots of Intraweb controls use an utility function to render their HTML. This function called TextToHTML is implemented as a class function of TIWBaseHTMLControl class. This function is used because if you have an IWText component with a caption containing a special character, lets say, an ampersand (&) it must be properly encoded to be shown by the browser. An ampersand declares the beginning of an entity reference (a special character) in HTML, so to be shown in the page it must be replaced by "&" (without quotes). If you have IW sources you can search for TextToHTML and see that it is used in a lot of places, most inside RenderHTML and RenderAsync methods.

2) TextToHTML implementation

Well, TextToHTML is not as fast as it could be. If you imagine that a lot of strings should be processed by TextToHTML before they get to the browser you may improve your application performance if you make that function faster, isn't it? Let's look at TextToHTML code:
class function TIWBaseHTMLControl.TextToHTML(const AText: string; 
  const AConvertEOLs: Boolean; 
  const AConvertSpaces: Boolean): string;
var
  f : integer;
  xIsCallBack: Boolean;
begin
  Result := '';
  xIsCallBack := GGetWebApplicationThreadVar.IsCallBack;
  for f := 1 to Length(AText) do begin
    case AText[f] of
      '<'  : Result := Result + '&lt;';
      '>'  : Result := Result + '&gt;';
      '"'  : Result := Result + '&quot;';
      '''' : Result := Result + '&#39;';
      '&'  : Result := Result + '&amp;';
    else

      {$ifdef UNICODE}
      if (Char(AText[f]) in [#10, #13]) then begin
      {$else}
      if (AnsiChar(AText[f]) in [#10, #13]) then begin
      {$endif}
        if AConvertEOLs then begin
          case AText[f] of
            #10 : Result := Result + '< br >';
            #13 : Result := Result + '';
          end;
        end else begin
          Result := Result + AText[f];
        end;
      end else begin
        if (AText[f] = #32) and AConvertSpaces then begin
          if xIsCallBack then begin
            Result := Result + '&nbsp;';
          end else begin
            Result := Result + ' '
          end;
        end else begin
          Result := Result + AText[f];
        end;
      end;
    end;
  end;
end;
We can see that there is a main loop interating throught all characters of the AText string, concatenating char by char to generate the result. This approach has two drawbacks: (1) Every string concatenation requires a new memory allocation and (2) it is SLOW!

3) A New TextToHTML implementation

We can implement TextToHTML using another approach, used by Delphi's own RTL in functions like HTMLEncode (unit HTTPApp.pas):
class function TIWBaseHTMLControlHack.TextToHTML(const AText: string; 
  const AConvertEOLs: Boolean; 
  const AConvertSpaces: Boolean): string;
var
  Sp, Rp: PChar;
  xIsCallBack: Boolean;
begin
  xIsCallBack := GGetWebApplicationThreadVar.IsCallBack;
  SetLength(Result, Length(AText) * 10);
  Sp := PChar(AText);
  Rp := PChar(Result);
  while Sp^ <> #0 do
  begin
    case Sp^ of
      '&':
        begin
          FormatBuf(Rp^, 5, '&amp;', 5, []);
          Inc(Rp, 4);
        end;
      '<',
        '>':
        begin
          if Sp^ = '<' then
            FormatBuf(Rp^, 4, '&lt;', 4, [])
          else
            FormatBuf(Rp^, 4, '&gt;', 4, []);
          Inc(Rp, 3);
        end;
      '"':
        begin
          FormatBuf(Rp^, 6, '&quot;', 6, []);
          Inc(Rp, 5);
        end;
      '''':
        begin
          FormatBuf(Rp^, 5, '&#39;', 5, []);
          Inc(Rp, 4);
        end;
      '\':
        begin
          FormatBuf(Rp^, 5, '&#92;', 5, []);
          Inc(Rp, 4);
        end;
      #10:
        if AConvertEOLs then
        begin
          FormatBuf(Rp^, 4, '< br >', 4, []);
          Inc(Rp, 3);
        end
        else
          Rp^ := Sp^;
      #13:
        if AConvertEOLs then
        begin
          Dec(Rp);
        end
        else
          Rp^ := Sp^;
      #32:
        if AConvertSpaces then
        begin
          if xIsCallBack then
          begin
            FormatBuf(Rp^, 10, '&amp;nbsp;', 10, []);
            Inc(Rp, 9);
          end else
          begin
            FormatBuf(Rp^, 6, '&nbsp;', 6, []);
            Inc(Rp, 5);
          end;
        end
        else
          Rp^ := Sp^;
    else
      Rp^ := Sp^
    end;
    Inc(Rp);
    Inc(Sp);
  end;
  SetLength(Result, Rp - PChar(Result));
end;
This new implemenation has two main advantages over the former: (1) There is less overhead due memory allocations, and (2) it is FASTER! :)

4) The problem

TextToHTML is a public method of TIWBaseHTMLControl but it is not virtual. Even if it were virtual, we would have to create descendant classes to override this method and it is not acceptable.

5) The solution: Hack it!

Thanks to great code (RtlVclOptimize.pas) created and made available by Andreas Hausladen we can change TIWBaseHTMLControl implementation on the fly, patching the class. I will not enter in detail about Andreas code, but the hack is done replacing, in memory, the TIWBaseHTMLControl.TextToHTML method by another method from some other class. I needed to change RtlVclOptimize.pas a little, putting the declaration of the function "CodeRedirect" in the interface section, so it could be used by my code, outside that unit.
Update: I'm not using Andreas RtlVclOptimize.pas anymore because it has a few incompatibilities with recent Delphi versions. I'm using another unit CodeRedirect.pas, included in the download file.

6) Speed comparison

The use of Andy's RtlVclOptimize unit alone can significantly improve IW applications performance. Moreover, in my tests, my TextToHTML implemenation is more than 4 times faster than the original. Original code: 14.7 seconds versus 3.4 seconds using my code (1,000,000 function calls). Another advantage is less memory allocations for strings (less memory fragmentation and less LOCK+multicore related issues - read more about it here).

7) More Hacks!

Well, not satisfied hacking TextToHTML code, I did the same thing with another TIWBaseHTMLControl method, TextToJSStringLiteral . Finally I've packed all these little things inside a unit. To speed up your IWApplication a bit just declare RtlVclOptimize (from Andreas) and my unit, IntrawebPatch.pas inside your .DPR file and you are done!

8) Finally, the downloads!

You can download my patch unit (IntrawebPatch.pas) and other required files here.

Enjoy!

Friday, January 13, 2012

Delphi 5 Update packs available again!

Sorry folks, my links to Delphi 5 Update packs were broken. Now I've fixed them :-) You may find my original post with download links here.

Tuesday, January 10, 2012

BDE and Windows 7, 32 and 64 bits

I know, I know... BDE is discontinued and stuff. But many people still need BDE running in new Windows 7 32 and 64 machines. Here is a working Setup for BDE 5.2.0.2 (the latest version released by Borland) plus all SQL Links.
Notes:
  • Installs BDE 5.2.0.2 + all SQL Links
  • During setup there is an option to remove previous BDE registry entries.If you have an old corrupted BDE installation, maybe this option can fix it (It will remove EVERYTHING below HKEY_LOCAL_MACHINE\SOFTWARE\Borland\Database Engine registry entry, and you have been warned!).
  • This setup was tested and works perfectly under Windows 2000, 2003 Server, XP, 2008 Server, Vista and Seven, 32 and 64 bits.
  • The setup correctly prompts for UAC elevation. By the way, it was created using Jordan Russell's excellent installer creator Inno Setup (also created with Delphi)!

Download BDE 5.2.0.2 Setup in English:
http://www.alex7691.com/download/Setup_BDE52.zip

Download BDE 5.2.0.2 Setup in brazilian portuguese:
http://www.alex7691.com/download/Setup_BDE52_PTB.zip

Note: During last months the number of downloads increased a LOT. Some users were receiving messages like "bandwidth limit exceeded". Now I have upgraded my host service plan, so you can easily download it!Enjoy! :-)

Some user gave a false alert about malware/trojan/virus/whatever inside the files. According to VirusTotal (http://www.virustotal.com) this is completely untrue.
Latest scan report of this file: https://www.virustotal.com/en/file/7d37616710f8d5da8d2f859560814fd89b22aef7b6fb47902161bc9f7289145a/analysis/

In short: 0 detections!

Please feel free to re-submit it ;-)

PS: Rene Nussbaum, from Germany wrote me asking about the setup license model/type. Well... This setup is 100% free including its usage in commercial software. You can include this setup in your deployment package, and you can also redistribute it.

Enjoy! :-)