MemoryManager revisited

Starting with DS version 1.1, there is now a new MemoryManager available, which was almost written from scratch. My past experience showed that the former implementation was somewhat limited, as you were forced to preallocate a fixed amount of memory prior to your application’s entry point. The new version now supports dynamic memory allocation, so memory consumption grows with application demand (you can still define an upper limit to be on the safe side). It’s now also possible to resize memory space after it has been allocated. I’ve also simplified the API and optimized the code.

Using alchemy memory in HaXe is now as simple as writing:

var integerArray = new IntMemory(x);
integerArray.free();

The first line allocates space for storing x 32-bit integers, and the second line will free up used memory once it’s no longer needed. The same also applies to BitMemory (a bit vector), ShortMemory (16-bit integers), FloatMemory (32-bit IEEE 754 single-precision floats) and DoubleMemory (64-bit IEEE 754 double-precision floats). Each implementation provides a get/set method to read/write the value from/to index i (I hope HaXe gets support for overriding [] in the future so we can omit the get()/set() methods); an offset property that stores the memory offset in bytes and a bytes property indicating the number of allocated bytes. Custom classes can be realized by inheriting from the abstract MemoryAccess class (yes, HaXe has private constructors ;-))

Frequent allocation/deallocation leads to fragmentation so the user can request defragmentation by calling MemoryManager.defrag(). In addition to the defrag() method, MemoryManager.pack() frees up unused space so it can be garbage collected. To be exact: All existing bytes are copied from the current ByteArray accessed by the alchemy memory opcodes to a smaller ByteArray, and once the reference to the former ByteArray is GC’ed the former content is deallocated (setting the .length property of the ByteArray seems to be a bit flaky and sometimes results in a runtime exception).

A quick note about MemoryManager.BLOCK_SIZE_BYTES: This value defines the growth-rate of the memory. The default is 1024 bytes (must be a power of 2). Every time the MemoryManager runs out of memory, a multiple of this block size is added. The block size affects performance and storage efficiency. As a rule of thumb the block size should match the average size of all ‘memory arrays’ used in the application. Using a tiny block size when storing 2k images is obviously a bad idea.

A major advantage of using alchemy memory is that you can use the appropriate data size that fits your needs which brings down memory usage. For example if you know in advance that your numbers are in the range 0…100 you can use bytes, or if you need floats without full 64-bit precision, you can fall back to 32-bit floats. In AS3, you are constrained to 32-bit integers or 64-bit double precision floats. Using the ByteArray is not an option because it’s too slow.

Performance

Although the preview version of FP 10.1 slowed down memory access a bit (it seems that the latest release fixes most performance issues) using those special alchemy memory opcodes is by far the best way to work with numbers. Time for some real world examples!

Example 1: JPEG encoding

After porting the JPEG encoder from the Flex framework (mx.graphics.codec) to HaXe I’ve encoded an empty 1024×786 bitmap and compared the numbers:

The HaXe version encodes the image in about 100 milliseconds, the AS3 version takes almost a second.

Example 2: Bit vectors

My next test compares an AS3 bit vector implementation from generalrelativity.org with the BitMemory from the ds package:

Combining alchemy memory, optimized byte code and inlining gives some excellent results :)

Example 3: FP10 drawing API

The last test draws triangles with the FP10 drawing API and measures the time needed to build a GraphicsPath object from a command and a data vector (without calling graphics.drawGraphicsData()). The HaXe version uses my VectorRenderer class which utilizes alchemy memory as a temporary buffer.

All results are based on the windows release player 10,0,42,34 and use the MemoryManager class.

Traversing the display list

A great HaXe feature is that you can define your own Iterator and execute it with the for-syntax.
It can be used in many different ways and drastically improves readability of your code. AS3 developers often need to look at the display list, so I wrote a basic DisplayListIterator to handle this task. Here is an example:

using de.polygonal.gl.DisplayListIterator;
...
for (i in Lib.current.stage) trace(i);

This will print out all display objects in the display list.

If you are not familiar with HaXe, Lib.current.stage points to the MovieClip of the Document class, and the using statement automatically creates a DisplayListIterator whenever it is called on a DisplayObjectContainer. So the statement ‘Class.method(arg)’ is transformed to ‘arg.method()’. Without ‘using’ I would have to write:

import de.polygonal.gl.DisplayListIterator;
...
for (i in new DisplayListIterator(Lib.current.stage)) trace(i);

Let’s finish with a simple example that changes the text color of all text fields to red inside a DisplayObjectContainer.

var container:Sprite = myTextFieldContainer;
for (i in container)
{
  if (Std.is(i, TextField))
  {
    cast(i, TextField).textColor = 0xff0000;
  }
}

Useful, isn’t it :)

Font rendering with the FP10 drawing API

After all the vector/font rendering library is done - the HaXe sources and SWC files for ActionScript 3.0 are available on the polygonal google code project page. As a reminder - the project started as an experiment if it’s possible to render fonts using the FP10 drawing API without loading or embedding any additional assets.

For obvious reasons, I can only include free fonts. At the moment the de.polygonal.graphics.text.fonts package contains Microsoft’s TrueType core fonts hosted on sourceforge, the Bitstream Vera fonts as well as the famous bitmap04 pixel fonts.

The pros and cons

Pros:

• No font embedding required :) Import a font class and you are ready to go
• Provides high quality font rendering; best used for extra smooth text animation
• Seriously fast!
• Seamless integration into the FP 10 drawing API

Cons:

• An ASCII set of printable characters adds about 20kb-30kb to the swf file - I’ll try to reduce this in a future release.
• Not very readable at small font sizes (except pixel fonts) because it does not include any hinting information for improving the quality - text remains legible down to about 12 points (viewed at 100%)
• You need a copy of Fontographer 4.1 to convert ttf files
• No text field functionality yet
• Only supports ASCII character (latin character set like ISO-8859 is planned)

Examples

Here are the ‘MS core fonts for the web’ rendered with the font library:

This also works amazingly well for pixel fonts (which I didn’t expect at all):

How it works

First a .ttf file is loaded into Fontographer and exported as a postscript file (I tried other methods but I stuck with this approach because postscript files are easy to understand). A parser reads this file and generates a HaXe class that contains the glyph data. The result is something like this: Arial.hx.

If using HaXe, the font_inline compiler flag gives you control over compilation time vs runtime performance. If omitted, compilation is fast so it’s best suited for frequent testing and debugging and the compiler-based auto-completion remains responsive. If compiled with -D font_inline, compilation is slow but results in the best performance.

The actual rendering is done by a class named de.polygonal.graphics.VectorRenderer. It uses the FP 10 drawing API in conjunction with ‘alchemy memory’ as a temporary buffer to gain some extra speed. At first all drawing commands are written into a chunk of memory, then copied into a vector and finally sent to the screen via graphics.drawGraphicsData(…). Depending on the CPU this is roughly 1.5-4x faster than using only vector. Note that this only accelerates the process of preparing the data, not the rendering itself (everything beyond drawGraphicsData())

ActionScript 3.0 usage

Grab the SWC file, add it to the library and the following code should (hopefully) compile fine:

package
{
  import de.polygonal.ds.mem.MemoryManager;
  import de.polygonal.graphics.text.fonts.coreweb.Arial;
  import de.polygonal.graphics.VectorRenderer;
  import flash.display.MovieClip;
  import flash.Boot;

  public class Main extends MovieClip
  {
    public function Main():void
    {
      new Boot(this);
      MemoryManager.allocate(4096);

      var vr:VectorRenderer = new VectorRenderer(512);
      vr.setLineStyle(0, 1, 0);

      var font:Arial = new Arial();
      font.bezierThreshold = 0.001;
      font.setPointSize(100);
      font.setRenderer(vr);
      font.write("Hello World!", 0, 100, false);

      vr.flush(graphics);
    }
  }
}

The source code reads like this:

• initialize HaXe specific things
• allocate 4 megs of alchemy memory to be on the safe side
• create a vector renderer using a buffer size of 512kb
• assign a line style (rgb, alpha, thickness)
• create a font object
• define curve smoothness, the smaller, the better (0=linear approx. using 2 segments/curve)
• set the font size: 100 equals 72pt or one inch.
• assign a renderer so the font can send drawing commmands to it
• draw “Hello, World!” at the coordinates 0,100 (x,y), if the last parameter is true, the text will be centered around (x,y)
• flush the buffer which draws everything to the screen

Glyph and text bounds

You can compute axis aligned bounding boxes for the whole text block or individual characters using the getBounds() and getIndividualBounds() methods prior to drawing the text:

Creating font classes

Converting fonts can be done using de.polygonal.graphics.text.util.EPSConvert.

New data structures library released

I’ve just released the first official version of ‘ds‘ (aka data structures), the successor of as3ds which is written in the HaXe language. (The name hx3ds was used during development and is obsolete because ds is now a module of the polygonal library).

The library is found here: http://code.google.com/p/polygonal/. There is also wiki page describing how to use it with ActionScript 3.0 (there are still some issues with the HaXe generated SWC files, I hope Nicolas finds some time to address them in a future release).

What’s new

The new release (changelog) contains many bug fixes, new features and a refined documentation. While there are still many things on my TODO list, I think the library is stable enough so it deserves to be the first major release. And since I’m using the library in all of my projects, you can expect some regular updates.

Why the heck HaXe !?

Firstly, ActionScript 3.0 doesn’t support any kind of generics/templates like Java does for example. The typed vectors introduced in FP10 are too crippled and limited so in the end you are still stuck with dynamic containers, which is a bad for complex projects. HaXe supports type parameters - in the image below you see that I’ve created a graph of stacks of hash maps and everything is typed. The nice thing is that you still get auto-completion:

Secondly, HaXe provides a lot of syntax sugar that sweetens your daily coding experience. As an example, consider iterating over a collection. In HaXe it’s just:

//iterate over all elements of a 2D array:
var a = new Array2<Int>(3, 3);
for (i in a) trace(i);

In AS3 you have to create an iterator object and explicitly call the next() and hasNext() methods:

var a:Array2 = new Array2(3, 3);
var itr:Object = a.iterator();
while (itr.hasNext())
{
  var element:* = itr.next();
  trace(element);
}

In general, HaXe implicitly does a lot of things for you! Probably one of the best language feature is type inference/implicit typing, which is absolutely brilliant for quick prototyping. It’s like writing AS1 but without compromising speed and type safety. At first it’s hard to get used to it because all ActionScript text books never get tired of repeating how important typing is and that everything should be typed. But If you think about it, the compiler should handle it where possible and relieve the developer from writing clumsy types over and over again:

//AS3
var n:Number = 1;
var i:int = 1;

//HaXe
var f = 1.0; //compiler infers that n is a float
var i = 1; //compiler infers that n is an int

Thirdly, because as3ds was all about efficiency I couldn’t resist HaXe because it’s much faster. From my experience, ds performs ~2-6x better than as3ds. Same runtime, huge difference!

Updates

After a long silence it’s time to update my blog again. So what happened? I moved from AS3 to HaXe and actually published my first flash game written in HaXe. This was a very ambitious job and virtually took all my time because in addition to a very tight deadline I had to port my AS3 libraries to HaXe.

By the time the game was finished I started to work on a single open source library called polygonal as a replacement for the as3ds and motor2 project. The latter simply became unmanageable as it was using many other small libraries - so it made sense to bundle everything and provide a single out of the box solution, rather than creating a bunch of small projects.

Meanwhile, the polygonal project is hosted here: http://code.google.com/p/polygonal/
The next task is to update the physics engine so it compiles against the new library.