Problem with System.Windows.Forms.WebBrowser

I used a lot System.Windows.Forms.WebBrowser in my LSSaver project to process the pages. Recently I got a feed back that LSSaver stops to load the blog titles and is blocked there. The problem seems to happen on a WinXp + IE6 platform.

After investigation, the problem comes from a null pointer exception, but why there is a null pointer? A step by step debug showed me that when using WebBrowser’s DOM to retrieve certain HTML element, the HtmlElement.NextSibling returns an invalid pointer even when there is no more sibling of the calling HtmlElement object. It seems that this is a known issue in the .NET framework:
http://connect.microsoft.com/VisualStudio/feedback/details/304466/htmlelement-nextsibling-returns-incorrect-value

Anyway, I won’t rely on the NextSibling is null or not, I get the number of children from the parent, then rely on this count when traversing all the children elements. The problem is fixed.

Overloading operator << for containers

Today, I saw a line of code in one of our cpp files, it looks like this:
marshaller << obj_list;
what interesting is that obj_list is an instance of type std::vector, and one of my colleagues asked me if he can do similar things with an instance of type std::map<>.

The answer is "I don't know", because it depends on if the operator << is overloaded for the marshaller type for std::map<>.

The secret why "marshaller << obj_list;" works is that, the marshaller type has an overloaded operator << for std::vector. Something looks like this, assuming marshaller has type T:
T  &operator << (T &t, const vector avector){...}

Here is an example with cout:
ostream &operator << (ostream &out, const vector avector)
{
    vector ::const_iterator itr = avector.begin();
    for (; itr != avector.end(); ++itr)
    {
        out << (*itr);
    }
}

int main(void)
{
    vector str_vector;
    for (int i = 0; i < 10; i++)
    {
        stringstream str_stream;
        str_stream << i;
        str_vector.push_back(str_stream.str());
    }
    cout << str_vector;
}
The output is:
0123456789

So for this works with std::map<>, we need to overload << for std::map<>. 

Finally, it turned out that there is no such an overload, unfortunately. So we have to do it for each pare in the map.

Do It in a Programmer's Way

DiPw, I'd like to pronounce it like "deepyou", is my preferred way of doing things, especially things related with my job as a programmer/developer.

Recently, I was assigned a job to migrate documents from the old system to the new one, and this is to be done for each of our customers. In order to secure the migration, I need to check many configurations on both systems, they need to be similarly configured so the migration is transparent for the customers, ideally. What complicates things is also we have test systems and production systems, and the migration need to be done first on test systems then on production systems. So as a result, I have a lot of things to check and compare. Even worse, the settings are managed by several GUI applications so I have to click many times to find one setting on the old system then, do it on a different GUI on the new system. That is not only boring but also fault prone.

A colleague of mine kindly told me that the content of the GUI applications are finally stored in the database and he also gave me some SQL to retrieve the values. That is a big help, however, I still have to manage two types of databases, Oracle and SqlLite.

Why I have to do it manually? It's time consuming and error prone. Why not write a program doing all of these jobs for me? With always DiPw in head, I decided to write a Python program doing these boring jobs and finally it took me one day. Voila, now with the program, I retrieve all need information on all systems for a given customer in 20 minutes and when needed, I can retrieve most up-to-date information whenever I want in 20 minutes, this is impossible without the program. And when it comes the time to migrate another customer, I just run the program for this customer.

Well, I admit that the job is not all about getting all the settings, but it is the most boring part, so let a program do it!

A Real-life Story about Bad Type Cast

We have a problem today in our soft. That is, in short, we have 285 lines of text to display but only 231 lines are displayed. The investigation is not so easy because the text to be displayed is converted from structured message and the number of lines are calculated on the fly.

Basically, there is a segment in the message that specifies the maximum allowed number of lines, the sender of the message has set it to 999, assuming it the maximum possible lines in the message.

So if you are really a geek in programming C/C++, you may already know where is going wrong. Yes, an integer 999 when assigned to a unsigned char, the latter will have value 231. In more detail: binary presentation of decimal 999 is 1111100111, when assigned to unsigned char, we have only the lower 8 bits left: 11100111, it equals decimal value 231.

The problem is quite clear, but it has been hard to find it out because it hides itself deep in one of the libraries our soft uses.

Another interesting thing is that the library code uses atoi to convert the maximum allowed size in the message to an integer value, but with a parameter like this: "999blabla", that is the value in string followed by other alphabetic characters. atoi is designed to handle this correctly, take a look at the specification of atoi:

int atoi ( const char * str );
Convert string to integer
Parses the C string str interpreting its content as an integral number, which is returned as an int value.

The function first discards as many whitespace characters as necessary until the first non-whitespace character is found. Then, starting from this character, takes an optional initial plus orminus sign followed by as many numerical digits as possible, and interprets them as a numerical value.

The string can contain additional characters after those that form the integral number, which are ignored and have no effect on the behavior of this function.

If the first sequence of non-whitespace characters in str is not a valid integral number, or if no such sequence exists because either str is empty or it contains only whitespace characters, no conversion is performed.

LSSaver Official Blog Announced

I created a dedicated blog for LSSaver.

http://lssaver.blogspot.com/

LSSaver 1.0 Release, Free download.

Finally, I'm able to announce the release of LSSaver 1.0, the result of my two-month project LiveSpaceSaver.

As said, it is for saving your Windows LiveSpace Blogs.

If you are interested, you can google LSSaver or simply download it here:
http://byunsoft.webfreehosting.net/index.html

Bug reports are appreciated :D

My New Project, a LiveSpaceSaver

I paused a little on the DirectX development because the first objective had been achieved and also a technical obstacle that I cannot find a way to float a D3D window on the desktop.

Main while, I turned to a new project that aims to write a program to save the Windows LiveSpace blog and album contents. Actually, I wrote one a year ago but for Linux, now I'd like the same for Windows.

The function of the program is simple, it saves the LiveSpace blog and photos in the albums to local disk. The program works like a crawler, it obtains the html contents from the server and extract interested part from them, after all, I don't want to save the complex page with lots of useless links, banners...

This project will be a RAD. With .NET, I prefer managed C++.

Basically, I'd like it to be a GUI program, on the left, I'd like to have a TreeView that shows the blog titles grouped by months, photos grouped by albums, same as shown on the webpage. TreeView does not support multiple selection by itself, but one can write customized code to support that. For the first prototype, maybe using CheckBox is OK. On the right, I'll show the contents for selected blogs or photos. Then on the top of the main window, I'd like to have a strip that holds the input for LiveSpace username and the buttons for "Obtain" and "Save". So it is a very simple GUI doing simple things.

Obviously, I need to obtain the webpage from LiveSpace, I selected the WebBrowser control which is based on IE and is quite easy to use.
WebBrowser can parse the HTML into DOM, then we can use HtmlDocument obtained by WebBrowser::Document. In this way, it is easier to regenerate the HTML from the modified DOM to customize output.

As for the final step that is the actual save operation, I need to download the pic files appeared in the blogs as well as the photo files. I selected WebClient for the job.

Development is on going, first version coming soon. :D