Visual Studio .Net in the Real World

For all it's amazing features, the .Net framework isn't going to provide al the tools that you need to write your programs - and in particular, it's not going to go too far when it comes to developing multimedia applications. That is why, when looking at this new piece of software, we must also look at how well it interacts with other software, libraries and 'real-world' situations.

DirectX in the .Net languages

Particularly within Visual Basic, DirectX opens the doors to all forms of multimedia and is therefore extremely important. Unfortunately, the results aren't good. I had heard "on the grape-vine" already that there were some compatability issues involved in using the DirectX (for VB) type libraries in .Net, and upon testing them myself I found these stories to be true.

There are two aspects to this problem, firstly (and most importantly) the libraries themselves were not designed with .Net in mind (as far as I know, the .Net frameworks didn't exist when they wrote the libraries, so not their fault really!) and was custom designed for Visual Basic. Various methods in the library which never caused any problems with previous versions of VB are no longer valid through the .Net system - in particular the global D3DX functions dont exist. Secondly, due to Microsoft replacing the COM standards (with the new .Net frameworks etc..) you need to use the "COM Interop" services to access anything in the library - which adds a hefty speed penalty to every call. This effectively nullifies a lot of the reason for using DirectX to get high performance multimedia.

When I made an enquiry into the state of affairs (on the MS newsgroups) one of the Microsoft team clarrified that these problems had been noted, but there was not going to be any patch/upgrade to fix things. DirectX 9 will be using a .Net compatable managed code library, which probably means it'll need Visual Studio .Net to work, and should definitely work properly! In the meantime, you can only try using the functions that do work - but it's not easy with a lot of core functions missing or just not working. Direct3D and DirectPlay seem to be the worst affected. Luckily, if you're using DirectX through C++.Net you shouldn't have any problems at all (as I mentioned earlier, Microsoft cant really change the language).

Faster Code?

Faster code is the holy-grail of almost every multimedia/games programmer on the planet. Given the complexity of the code/algorithms routinely used in this field, performance is critical - in a game, the difference between 30ms and 60ms is the difference between playable and unplayable (30ms = 35fps, 60ms = 15fps). Performance was always going to be an interesting area - .Net is generally aimed at web services, server side applications and other similiar programs. These programs would probably require a broadband internet connection to operate properly, so whilst speed is important (not many people like waiting!) it is no-where near as important to the business applications as it is to a multimedia application.

I wrote 4 programs to test the speed, one in VB6, one in VB.Net, one in VC++6 and one in VC++.Net. Whilst each one was slightly different (due to different languages!) they were identical in the numbers, methods and functions; the idea being to test each languages respective compiler. Before you look at these results you need to bare in mind that these aren't the defining benchmark results for EVERYTHING - there would be no way (in this review at least) to judge the speed of everything. The raw results are in the following table:

System Used: Athlon 700mhz, 288mb SysRAM, 15.3gb HDD. (Tested again on 500mhz AMD K6-2, results followed similiar trend).
Test Conditions: Clean boot of WindowsXP for each version of the program.

Straight away you can see that the C++ times are much faster than the VB times, but this was to be expected (we all know C++ is faster!), however there is some pretty damning evidence straight away about .Net's performance. I have to say I was genuinely shocked when these results came out (and I have run them at least 5 times each on seperate occasions to check it is "constant"), even if .Net was going to be faster/slower I expected it to be a pretty close fight, not almost 1/2 the speed of previous (5yr old) compilers.

The times above are for a single call of that particular maths function (in nanoseconds), to get this result a loop with 1 million iterations was used (to find the total time) and then divided by 1 million to get an average time for each individual call. Whilst this will include some time used to execute the loop code, all 4 programs used this same method so any timing error will be in ALL times, thus wont affect any comparisons. Each test of 1 million iterations was called (back to back) 50 times and then averaged again - to make sure that the times weren't affected by any loading/initialisation slow-down. I am pretty certain that the method for getting these timing values is accurate, but if you wish to dispute them feel free, but you have to email me with some code that I can try for myself :)

Despite the poor results above, there are some odd exceptions to the rules - with more "complete" tests using pre-written code of varying types I would get faster results (particularly apparent when doing memory manipulation/array access), but they were limited to a maximum of 10% faster, however, I also got several runs of these tests showing that the previous compiler was faster... odd!

Having got these results I instantly went about trying to work out why they are so poor. Whilst it is unlikely to be just one reason, there is a very obvious thing to note - the compiler technology used. VB.Net no longer supports "native" Win32 EXE's (like VB6 did), only VC++.Net still compiled to proper EXE's. Instead, the compiler actually compiles down to an intermediary assembly like language - MSIL (Microsoft Intermediate Language). This code is then compiled as the program runs - using a JIT (Just In Time) compiler. As a result of this, programs written with .Net often take considerably longer to load up. To counter this, I ran each test 50 times back to back (as mentioned above), to make sure the JIT compilation wasn't interfering - the compiler will keep compiled functions in memory if they are going to be called regularly - thus if the function is called 50 times it is safe to assume that it wont need to re-compile it every single time. This lack of full compilation may well account for the rather poor times registered for VB.Net, however it shouldn't be a reason for the poor VC++.Net times. During my research into the matter, one individual suggested that the C++.Net compiler compiles to MSIL (like the other languages), but then does the runtime compilation part at compile time, thus you're left with a Win32 executable. I cannot find official confirmation of this, but if it is true then it could explain the poor results.

Portable Executables

As briefly mentioned above, .Net programs are compiled to an intermediary assembly language. This means the .exe files generated with .Net compilers aren't actually the same (file format/data etc...) as traditional .exe's from previous compilers. There is one huge advantage to this - platform independance. If the code is not specialised to a particular instruction set / operating system (Win32/x86 etc...) but instead, uses a JIT compiler while running, all that needs to be done to get the program running on ANY computer is to write a compatable JIT compiler. This means that we could very soon see Linux, BeOS, MacOS .Net frameworks - which means that you can distribute any .Net application to any of these systems. This removes some really big blockages in the distribution of software - companies/teams at the moment spend a long time developing software for 3-4 different platforms, whereas now they can target only one - the .Net framework. It also has forward-thinking in mind, the move away from Win32 (to Win64/Win128?) and x86 processors means that software will have to be changed to make use of this - if it is compiled to MSIL/.Net standards then all that needs to change is the JIT/runtime compilation module.

However, with all these great things it does leave for one slight problem. Hacking. A little program "Ildasm.exe" installed with Visual Studio .Net allows you to decompile any .Net executable to its MSIL source code. This isn't new - you can decompile standard Win32 EXE's to the assembly/binary data, however, ildasm.exe reveals a lot of information with it's decompiler - and where is it getting this information? it must be stored in the EXE! Whilst I'm no amazing assembly-level programmer, I can easily see how this can be used against the softwares creators. If you look at this decompiled sample, you'll see what I mean - it would not be too hard to work out any file-accessing code (such as that used for copy-protection/piracy) and "crack" it. Particularly given this powerful little tool - you can read the MSIL code, potentially change it, then recompile it again! (not that it needs full compilation though).

To read the last page click here or click here to return to "Learning to Talk the Talk".

• Introduction: Introducing the software, and the aims of this review.
• Getting Started With Visual Studio .Net: The installer, version, prices etc...
• The new IDE: New things in the Integrated Development Environment, and is it an improvement?
• Learning to Talk the Talk: Learning the new language (C#) and the changes to Visual Basic
• Visual Studio .Net in the Real World: Performance and real world capabilities
• Conclusion: Summing everything up in a neat way