PalOMoney 1.0.5 Released

After a few months and plenty of distractions, PalOMoney 1.0.5 has been released for testing. This version adds “direct connect” bank, credit card, and investment statement downloads and decent integration with Money. It also includes a patch utility app to fix defective Money software, as discussed in the wiki article Money Sunset Import Bug Remediation. As a public service, the patch utility is available to anyone as a free standalone download.

By design, the ofx statement downloads are driven by Money file settings that would have been used for the MS Money online services. Put another way, setting up a Money file for direct connect downloading involves setting it up as if Money were to do the downloading itself, instead of PalOMoney. This might just enable original Money 2008 (not Sunset) versions to resume downloads, but that will require more experimentation.

The statement download settings are much more complicated than the Money download substitute software currently available. The dialogs that control the settings are a user-interface nightmare, with dozens of controls, with a few modes thrown in. This kind of user interface design is not suitable for a non-technical user. No wonder the Money interface used so-called “branding servers” and databases to automatically configure their arcane settings! Because of this, the main settings dialog has an import and export capability that can be extended to automatically download settings from a web site or on-disk database, so that an easier user interface can be implemented. But it is still nice to have the “advanced” interface, with access to all parameters, for those who like to do it themselves.

The new PalOMoney Statement view window provides an interesting view into your finances. After download all your statements, the window shows the unreconciled entries (or statement items). Sorting this in inverse date order, newest on top, you can see what has just been charged to your accounts, without waiting for the monthly statement, which we don’t read anyway. This can make for some interesting conversations regarding where the money is going. The statement items disappear after you reconcile them in Money, so the list is only of the transactions of which you are not fully aware, and haven’t fully reconciled with your budget. I wonder if this new view will change any spending practices.

We still regret this business about fixing broken Money software. Because the Money source code is unavailable, a tremendous amount of effort is required to derive information using a debugger, which displays only the raw assembly language, if you’re lucky. It would be really great if some reverse engineer, professional or not, could take over this part of the project. Feel free to volunteer!

 

SayOk Auto-Closer

PalOMoney OFX Statement download has a Money import method that writes directly to the Money queue and notifies Money, bypassing that dreadful Money shell import handler that launches when you “open” an .ofx file, and requires you to click Ok for each download if Money is not launched. When Money is launched, an equally obnoxious offender is the Money “Import a file” dialog, which is a bit more pleasant on the eye but ultimately just as annoying. Who wants to wait an indeterminate amount of time, only to click Ok 16 times in a row? Something had to be done.

SayOk Auto Closer

The SayOk Auto Closer utility is a separate app that sits in your system tray and watches for a certain window title, in this case “Import a file.” When it notices a window with that title, it looks for a likely “Ok” button, and activates it, thereby closing the window. SayOk opens a notification balloon whenever it closes a window, and logs the closure in its display pane, and in a log file in your Documents folder. The app can be extended to look for other annoying windows.

See the wiki SayOk Auto-Closer page for up-to-date information.

PatchOMoney 107 Released

Microsoft Money Sunset users who experience crashes importing OFX files can download and use the new free experimental utility PatchOMoney from PalOMoney.com. The software comes with no warranty, and the user assumes all risks. The user must agree to back up their Money installation before using the software.

To download the software, start at the wiki page https://www.palomoney.com/wiki/index.php/Money_Sunset_Import_Bug_Remediation. Read the article, then follow the link to the  Money Sunset Patch Utility link. The download link is at the end of that article. Unzip the file, and follow the instructions. Good luck.

If this patch does not work for you, please post a message to the PalOMoney support forum.

Anyway, we can now go back to releasing PalOMoney 1.0.5, which was postponed prior to resolution of the OFX import crash issue.

Money Sunset Import Bug Remediation

Introduction

PalOMoney version 1.0.5 release candidate build 893 was being tested with Money Sunset on a 64 bit Windows 7 machine. While testing the Statement view Update Now function, it downloaded about a dozen statements, each time calling the (annoying) Money Shell Handler to queue them for processing in Money. After 12 Ok clicks, the downloaded and log files were examined and found to be Ok. However, when Money Sunset was launched, it crashed after a few seconds.

PalOMoney was used to repair the Money file outside of Money. Money was launched again, and suffered another crash. There was no way to prevent a crash. However, the Money file worked fine on another machine. After renaming the Money file, and opening an older file, the crash still occurred. The Money Sunset environment itself had become unstable.

This was not an auspicious beginning for PalOMoney version 1.0.5 release candidate build 893, and delayed the release of 1.0.5 pending a resolution of this issue. Ordinary users cannot tolerate this kind of performance.

This topic also has a wiki version, which will be kept up to date.

Research

It wasn’t really a problem with PalOMoney. There is a known problem importing OFX files into Money. Referring to Microsoft’s Raymond Chen’s Microsoft Money crashes during import of account – MSDN Blogs and Windows 8 64 Bit Version Compatibility with Sunset Money/. The 4 byte patch in the articles was applied, but Money still crashed. More research turns up MS Money Sunset+Windows 8, Crash when adding new Payee, which has a similar problem.

In order for Microsoft Money Sunset to be viable for ongoing use, it will require patches to fix bugs possibly introduced when downgrading Money 2008 to produce Sunset. This task has been (reluctantly) added to the PalOMoney project, but it adds the task of reverse engineering to the project.

Reverse Engineering to Correct Bugs

In the words of commenter to Chen’s blog entry above, “Reverse-engineering is a violation of the Microsoft Money EULA…,” and PalOMoney.com does not want to run afoul of that issue. Another commenter wrote that “There are certain court rulings about warranty of merchantability that override the EULA in situations like this one.” Although this comment is not a duly prepared legal memorandum that can be relied upon safely, PalOMoney.com has done the necessary examination and identified a bug and has developed workarounds and remediation.

Any readers with insight into this issue are encouraged to add it to the wiki topic or discussion, blog comments, or discuss it in the forums.

OFX Crash Bug Cause

The fault occurs when doing a string compare on uninitialized memory.

Apparently, the bug is caused several functions up the call stack by failing to initialize in all circumstances what seems to be an in-memory hash table. The initialization occurs in two phases. First the table is initialized with zeros to indicate no entries. Then an entry is made into the table. Each phase tests a pointer in a large object supplied by the calling function, and either skips or performs the initialization. When the system crashes, the first test skips the initialization. A call to the hash table find function then calls string compare on an entry in the uninitialized hash table, which causes the fault.

The crash occurs with OFX downloads from certain accounts at certain financial institutions. However, not all downloads of those accounts cause a crash. The cause of this is being investigated and will be reported here as answers are discovered.

Money Sunset crashes when processing the “Import Files” queue. This queue is not part of the Money file, so it will cause a crash when opening any Money file on that machine.

The “Payee Change” crash mentioned in the above blogs has not been investigated. It may be caused by the same problem.

OFX Crash Bug Workarounds

One workaround is to not import transactions from problem accounts at problem institutions, or to import them using Money’s File Import, and NOT by double clicking them and NOT using the Shell Open method to process downloads via PalOMoney, both of which add the downloads to the Import Files queue. However, this does not help very much when a user’s Money Sunset installation is unstable. So the first thing to do is to fix the installation, below.

The Money file is left with open objects by Money crashing. This is fixed either by opening it with Money (which repairs with a “Working…” progress bar, or by running the the PalOMoney File | Repair | Repair function.

The “Import Files” queue is saved in the registry under [HKEY_CURRENT_USER\Software\Microsoft\Money\17.0] “Import Files” key. The MULTI_SZ value contains the names of temporary files copied by the Money shell handler. The key is deleted (not there) when the queue is empty.

You can just delete the key, but PalOMoney adventuresome explorers are encouraged to rename the entry instead of deleting it.

  • Open RegEdit. You may not need administrative privilege to access HKCU. Try it. If you do, run RegEdit as administrator.
  • Locate the “Import Files” key, and press F2 to edit the key name.
  • Add a space and “00”, “01”, etc. to the key name (e.g., “Import Files 02”).

This leaves the files in your TEMP folder (e.g., \users\UserName\AppDate\Local\Temp), which are names with a “~of” plus a random number then .tmp (e.g, ~of896.tmp). You can ignore them for now, and use them for testing the remediation, below.

PalOMoney is being enhanced to include these functions. Also, PalOMoney will offer a direct queue import method, which write to the registry key directly, bypassing the annoying Money Shell Handler with its “Launch Money Now?” nagging.

Finally, the investigation into why some downloads crash and others don’t continues, and may result in PalOMoney being able to “filter” a download to add required or remove problematic OFX entries to prevent the crash in the first place. However, this would not fix the general problem of crashing when importing or changing payee names, etc. mentioned in the blog posts above.

Ofx Crash Bug Remediation

The Money Sunset software can be patched to eliminate the first test that skips the hash table initialization by replacing the test and jump with NOPs (we regret not having a more elegant solution). The exact dll location has to be calculated, and will be published along with a simple .exe file to apply the patches Money Sunset Patch Utility here when known.

The PalOMoney application cannot perform this function, because patching a program file requires elevated administrator privileges, and it is undesirable to run PalOMoney or any ordinary application with elevated privileges.

PalOMoney is working on a patch utility, which will be made available on this site to anyone without charge. The utility will require administrator elevation to run. Whether the Money Sunset software is under Windows file protection is unknown, but Chen’s article leads one to conclude that the software can be patched, and not automatically revert to an earlier version.

Cyber Wars

By now everyone has read about the hackers infiltrating the New York Times and Wall Street Journal computer systems and apparently obtaining confidential reporters notes, source information, etc. in order to intimidate existing and potential news sources. In this case, a government is fighting stay in power, the stakes are high, and the methods are both disturbing and likely effective. This is war, and the targets are high value.

But there is another war going on, and this one is all about money. This web site, from nearly the day it was opened for construction, long before it had been publicized in any way, has also been under attack. These attacks are coming from web bots: automated programs that scan the web for sites and attempt to exploit them. Web masters have to stay vigilant and react quickly to stay alive. It might be a fascinating spectacle if it weren’t so creepy, and outrageous. After all, we are supposed to be working on, oh, what were supposed to be working on before this web site? Now we have to work on just keeping the site from being hijacked. Multiply this by millions of web sites, and what is the productivity effect world wide?

At this point, the abuse and outright criminality is mostly about spamming. Traditionally, bots scoured the web collecting new email address to spam. Today’s spammer wants to improve search results for and visits to his web sites. These sites contain minimal original or useful content, and exist for the purposes of generating advertising revenue from the ads on their pages. Other client sites might be legitimate sites that have hired a “search engine optimization” consultant to increase their search ranking, and spamming is one of his methods.

In any case, the spammer attempts to log onto a site’s wiki and forum and post inane (e.g., Great post, Jack. You sure know how to write!) messages containing one or more links to client sites. The Google (or other search engine that uses a link-based page ranking scheme) web bot reads the post, and records the link. The more links to a site from legitimate sites, the higher the client’s page rank, and the higher the chance that the client’s site will show up at or near the first page of a search. Very simple, but the result is a nightmare for legitimate sites everywhere. Its an ugly reminder that some people are willing to inflict misery on others as a living, and there are so many of them. It’s massively wasteful of everyone’s resources, time, and energy, but they do get paid, and that’s what matters. Don’t forget that Google gets paid, too.

For now, this site is safe in its obscurity, zero page rank, all pages certified SSL, and the other security measures in place here, which pretty much foil the bots, famous last words. However, should the site become popular, humans will aid the bots and the site will have to implement yet another level of defense.

Free Advice

For the umpteenth time: change your passwords to strong passwords, and keep them different for every critical site. Consider using different user names for every site. Save these user names and passwords in one or more strong encrypted “password vaults.” There are many of these available for free. To combat email spam, get an email account where you can have many separate email address aliases, and use a different email address alias for every site. That way, you can identify and shut off the source of any spam you receive.

Although this is not an issue for Money users, do not trust your identity including important site user names and passwords to a source (e.g., an financial services aggregator) in the cloud. They are potentially even more insecure than an ordinary web site. While most sites store passwords as irreversible MD5 hashes, and the password cannot be decrypted, aggregators store passwords with reversible encryption in order to be able to supply the password to log onto your critical financial accounts. This, along with the large number of identities stored, makes them tempting targets for criminals and outright espionage. It’s only a matter of time before whatever defenses the aggregators have are breached, and then your accounts are at risk. You will never know until it’s too late. The articles on the web are hopelessly skewed and overly optimistic about your security at the hands of these sites, which makes sense, since the aggregators have an interest in making sure that potential customers read mostly positive comments about their sites.

Mantis Support Portal

After much fooling around trying to figure out how to get Bugzilla to work without exposing real user email addresses, we installed the Mantis bug tracker as the support portal. Like almost all other web applications, Mantis uses user-defined handles for login, and hides real names and identifiers, keeping the site safe from those awful spammers.

Bugzilla is really good in that it supports 3 levels of classification: product category, product, and component. This model fits the PalOMoney development and support schema perfectly.

Mantis uses projects and sub-projects instead, organized to any level, which is also cool, but not quite as helpful here. More importantly, Mantis requires editing .php files to customize the look and feel of the site: there are no themes, skins, or templates to customize the web app. Perhaps that’s why other sites using Mantis look like the out of the box Mantis. But, all in all, Mantis looks to be our best choice. If you have any other ideas, please leave a comment.

We will continue to work on Bugzilla (with what time?), but for now we are lighting up the new support portal powered by Mantis.

Web Site Woes

This was supposed to be a couple of days’ project, setting up a new web site to handle the PalOMoney project. Well, how time flies! Time is up, and the site is still not complete. It’s going to have to go live, anyway, and our apologies to users who will have to suffer with it.

For those of you who are not familiar with web site construction, it is possible to subscribe to a hosting service and implement a web site with just a few clicks. Oh, happy life! However, this site is not a hosted service. Instead, it is on its own server in the cloud, which means that everything is possible, and everything can go wrong. Having dedicated servers is necessary because one of the purposes of the site is to provide substitute quote feeds to MS Money users. But no one here has ever done web site or public server administration before!

This site consists of a WordPress blog, a MediaWiki wiki, a store for download control, a Phorum forum, and a Bugzilla issue (or bug) tracker for support. These portals are powered by popular, well respected software that nearly every web user has encountered on many other web sites.

Prior to this moment, the most critical web site issue was integrating the various portal logons into a unified site logon that would work with all the portals. It was the top priority project for the web site.

Now there an even worse problem, which has triggered this post. It is the new Bugzilla issue support portal, intended for use as customer support portal. The idea is to combine support tickets, bug reports, and enhancement requests plus road map reports into one web application, and Bugzilla does most of that beautifully. However, there is a fly in the ointment: Bugzilla requires site login using the user’s valid email address as the user name. This email address is viable to anyone who visits the portal. This is horrible if you care about spam not to mention privacy. Here is the text from the “create a new account” page:

“To create a PalOMoney Support account, all you need to do is to enter a legitimate email address. … PRIVACY NOTICE: PalOMoney Support is an open issue tracking system. Activity on most issues, including email addresses, will be visible to the public. We recommend using a secondary account or free web email service (such as Gmail, Yahoo, Hotmail, or similar) to avoid receiving spam at your primary email address.”

This is a ridiculous requirement. We refuse to compromise any user’s privacy and email to the merciless spamming web bots, secondary email account or not. This makes the portal unusable. We will have to go live using the forum for support, and lock everyone (especially spammers) out of the support portal until this is resolved.

Prior to this, we tried Redmine, which is too much a project manager. We now have to evaluate Trac or Mantis. But it looks like this project has to be pushed into the future, because it is impacting the site’s core purpose, which is, lest we forget, to develop the alternatives to MS Money.

Anyway, although it has been great fun playing system administrator on the site, it is now time to recruit a web master who can take this over, and make things right.

Web Site Development?

This business of trying to get Microsoft Money to work after “sunset” is taking a lot more time than expected. It’s just one thing after another. Today, it is web site development. Why? Well, I was afraid that Money would really become obsolete without a truly viable online update function. After months or researching and trying alternatives with varying degrees of success*, I drifted into creating a product myself, and it worked. I named the product PalOMoney, and have been using it for several months without mishap. Now I need to figure out what to do with this product to keep Money Sunset viable in the world.

As time permits, I plan to contact the other interested developers and bloggers and ask them to test the software. Some of these are interested parties with solutions of their own. Perhaps they can integrate my products into theirs. Certainly, I would like to integrate their products or workflows into PalOMoney. One problem is that these other projects are open source, but PalOMoney is not as it uses proprietary software. Also, some means for funding maintenance and development have to be determined. So the website is being developed to support distribution, testing, and support. There is even a 1-product store to support customer identification and downloads.

Another side of the Money functionality issue involves the Money central server architecture. It may be possible to provide links to resources just as the Money servers did by redirecting existing software to this or another site. So that is being explored here, as well. But, hey! Linux, php, cloud computing, all of this to support what seemed at the time to be a rather simple project.

 

* Various developers have tried various approaches, and their effort has resulted in software and procedures to update Money data files with bank and securities information, nearly achieving the functionality of Money 2008 with online services. However, I just dread the command prompts, downloads, updating, etc., that these approaches use. I learned python (a great language and system) to customize and further automate the bank statement download process for myself, but the result is still too much of a kludge, and the online quotes .ofx and dummy account method I find just unacceptable.