Showing posts with label odbcexpress. Show all posts
Showing posts with label odbcexpress. Show all posts

Wednesday, 9 January 2013

Migration from Delphi 6 to Delphi XE3

Welcome back! It's been 2 and a half years since I last posted on this blog. Although it is partly down to the usual "too much work" and "kids keep me busy", it's largely because even after all this time, I'm still on Delphi 6(!), and so had nothing new to report. That's right, we never managed to find the time to migrate to Delphi 2009.

Not for much longer though. We've taken the decision that in March/April this year, myself and the development team are going to take the plunge, and finally upgrade from Delphi 6. Not to Delphi 2009 as I originally wrote about though. This time, we're going straight to the very latest version, Delphi XE3.

This has come about not only because our hand has been forced by some of the third-party components we use (TMS Component Pack, ReportBuilder, Raize Components) - none of the new versions of these support Delphi 6 - but also because we're starting to think about mobile development. Delphi XE3 offers decent distributed database architecture with DataSnap which we can make use of to write some terrific companion mobile apps for our software.

It is somewhat of a relief to see that dbGo (ADO) is still well supported with Delphi XE3, vindicating the decision I made a few years ago to switch over to that from ODBCExpress. This will form the largest part of our migration process, as we have to move all existing components over to the dbGo equivalent. However, we have established how to do this without any real complications, and so it will just be more time-consuming than difficult.

Of course, there will still be the Unicode effect that came when Delphi 2009 was released, "breaking" existing code. But this has been well documented by Delphi luminaries such as +Bob Swart, +Marco Cantù and +Cary Jensen, and so this will present no real issues.

The superb ReportBuilder Enterprise by Digital Metaphors stores the database connectivity information within the report file itself, so I have already written a utility that replaces the ODBCExpress details with that of ADO. This will instantly convert our 250 standard reports, as well as the countless reports written using the built-in report writer by users for themselves.

The largest task at hand is converting our "plug-ins". When I first joined the company over 14 years ago, my task was to rewrite the software to convert it from Unix to Windows. There were multiple versions of the software for each customer, based on their bespoke requirements. I vowed this would not be allowed to happen to the Windows version of the software, and so developed the plug-in architecture which allows bespoke functionality to appear within the core software, as if it were standard. To this day it works perfectly, however its success has grown much larger than I ever anticipated, meaning there are now over 100 plug-ins for various customers and other "extra" features that we can provide. Each of these is a separate Delphi project, and so this will be enormously time consuming to migrate these.

But it will be worth it. We will finally be on a development platform that can provide us with "modern" features, whilst giving us opportunities to develop in other areas as well.

I'll keep posting as I come across any interesting areas during the move! I'd love to hear from you if you have done anything similar and have any "gotchas" you'd like to share.

Tuesday, 26 January 2010

The road to Delphi 2009 - Part 1 (v) revisited

Well that was judicious timing (for once)! Just as we decided to take the plunge and do away with ODBCExpress in favour of dbGo, I had an email from Pieter Myburgh at Korbitec. In it, he stated that the management had rejected the proposal to make ODBCExpress open-source and instead have "end-of-lifed" the project.

Now, Pieter has been nothing but helpful and this decision was out of his hands. But what a strange decision. After his personal email to me, another one followed to other developers which was a bit more definitive:


As you know a few of us here at Korbitec have been pushing to get ODBCExpress open-sourced, since there is no ongoing development, and recent changes in Delphi 2009 of the char & string types to Unicode have made ODBCExpress unusable in its current form in this version of Delphi. The bad news is that open-sourcing ODBCExpress has been rejected by management for various reasons. However the good news is that the latest source code is available for download by all of the existing registered users of ODBCExpress. Furthermore you are allowed to modify the source code in any way for your own purposes, or collaborate with other registered ODBCExpress users to port it to newer versions of Delphi if needed. So unfortunately this is the end of the road of Korbitec’s involvement with maintaining ODBCExpress.


Pretty much the same message is available on the ODBCExpress website.

It's an odd decision not to open-source it, as TurboPower did so successfully with its VCL component sets all those years ago. They can't make any more money out of it, they're encouraging developers to collaborate and help each other, they're giving away the source code to registered developers who don't already have the source - yet they won't make the source code available to everyone so that the project might live on.

Bit of a two-fingers up to Delphi really. Pretty sad ending to a good set of components.

Tuesday, 19 January 2010

The road to Delphi 2009 - Part 1 (v)

A new year and time to make a decision about how we are going to move our ten year old codebase from Delphi 6 up to Delphi 2009. If you've followed my other posts in this "road to Delphi 2009" thread then you'll know that I hit a stumbling block when migrating a third-party component, ODBCExpress. This is a third-party database connectivity component that was well written, well documented and had good reviews and articles in Delphi Informant (which dates it a bit for those who remember that magazine).

At the time we chose it, our product was based on the Access database and we were looking to add SQL Server support, so we chose it as a good option to handle any database type. I explained this back in June 2007.

Unfortunately, the product was no longer actively developed from Delphi 2006 and despite help from its main developer to at least get it compiled into Delphi 2009, it didn't work properly thanks to all the Unicode changes.

Seeing as we bought Delphi 2009 in June 2009 and haven't migrated to it yet, a decision has been made. We are going to move all the code away from ODBCExpress, and to the dbGo ADO components provided as standard with Delphi (from Delphi 5 through to the current Delphi 2010).

Presciently, in June 2007, I also said this:

I've thought about moving to one of those bundled Delphi offerings in order to save myself the worry should ODBCExpress go tilt. But I haven't...because the database architectures offered with Delphi keep changing and I'm not sure I want to run the risk of changing everything over only to have it "deprecated" like the BDE.

Well the dbGo components are still going strong and they made the effort to Unicode-enable them for Delphi 2009, so I'm classing this as a low-risk option.

I wrote some test programs using the ODBCExpress components and then migrated it to ADO to see what conversion issues I'd have. I made notes and it wasn't as bad as I thought. I can now do everything I could do with the ODBCExpress components, even if I have had to write a function that does what a component used to do. If anything, it has been an interesting exercise in the back-end of SQL Server.

What was particularly pleasurable was doing all the work in Delphi 6 and then simply recompiling it in Delphi 2009 to see what happened. The answer: it compiled and ran just as it did in Delphi 6 (give or take a few string cast warnings in my own code which is part of the Unicode migration, and expected). Perfect.

For those who may ask why I'm not going for dbExpress instead (which is also standard with Delphi and developed by CodeGear/Embarcadero), it was a couple of things. It was initially my preferred choice but the extra DataSetProvider/ClientDataSet layer required to use it visually gave me the creeps, and I've been reading some frightening problems that people are having with it with Delphi 2010 and SQL Server. Plus the architecture of dbGo more closely matches that of ODBCExpress, facilitating the migration.

So now we have a path, a light at the end of the tunnel. It's not going to be pretty, and I know it will be tedious, but it will be another third-party component set that I can drop which, as I said in another posting in June 2007:

that the less third-party tools I can get away with using the better, purely for future-proofing purposes. If Delphi then provides that same component as standard in its next version, I'll use that one and thus reduce my dependency.

I'm now looking forward to the journey again.

Thursday, 26 November 2009

The road to Delphi 2009 - Part 1 (iv)

So it has been a while since I last updated on my progress with the migration from Delphi 6 to Delphi 2009. There's a reason. I'm still on Delphi 6 and I've got nowhere fast with ODBCExpress!

To be fair to its developer, Pieter Myburgh, it isn't his project anymore and he has been very good in keeping in touch with me and letting me know how the open source status is coming along. Very slowly, it would seem. He created a project on SourceForge and that is the only activity there has been. I think legal red tape is giving him a bit of a headache.

I know I'm not the only one really hoping we can get ODBCExpress moved up to Unicode-enabled Delphi 2009. But I'm starting to come under a bit of pressure now to look at alternatives. It is understandable - we paid a lot of money of the new version (as it was then) of Delphi, under my recommendation, only to find we can't use it.

I've looked at changing to dbGo (ADO) or dbExpress, and have participated in some interesting discussions on Stack Overflow about the merits of both. But ultimately, the thought of having to change to the core database connectivity components of 10 years worth of software (including nearly 50 plug-ins) makes me shudder. Not least because we use a great deal of the intricate little details of ODBC and ODBCExpress, which we'd have to find alternatives for - if, indeed, an alternative is available.

On the upside, we would have had to buy Delphi 2009 or 2010 this year anyway, because of Embarcadero's decision to only allow upgrade prices for users of Delphi 2006 and later from 2010.

And also I have got all the other component sets that we use installed in Delphi 2009, and the main application compiles. So small steps are being made.

Thursday, 9 July 2009

The road to Delphi 2009 - Part 1 (iii)

Still calling this Part 1 (Act 3) as I'm not actively pushing anything forward until I've overcome the ODBCExpress hurdle. I've had some positive steps forward with this.

I have been in regular contact with Pieter Myburg, its original author. Even though he is no longer actively developing it, he has very helpfully made some changes to 3 files that allow the packages to compile and install into Delphi 2009.

Even more helpfully, he has agreed to Open Source it and is looking for moderators. I have agreed to help (as has another user this morning) but we need more, as I can't honestly say with full confidence what I know what I'm doing with the code! I don't want to give out his email address on a blog but the newsgroup for ODBCExpress is news.korbitec.com.

So now I can connect my data and view it. I've already noticed that nvarchar fields (SQL Server) display with non-printable characters so that is my first area of investigation.

I have also come across another peculiar one. If I connect within the IDE, everything displays in a DBGrid. If I run the executable, when the database connects, the debugger pops up and shows me the CPU Window on ntdll.DbgBreakPoint. This used to happen to me in Delphi 6 when I pressed F12 in my software, which I understand was a bug left in the IDE. However, this happens when the database connects so I'm guessing it must be something in code.Whether that is ODBCExpress code or native VCL code, I'm unsure.

Slowly moving forward.

Wednesday, 1 July 2009

The road to Delphi 2009 - Part 1 (reprise)

So I haven't really got any further yet because I'm waiting to hear back from the author of ODBCExpress whether or not he will look to upgrade the packages for Delphi 2009. He's emailed to say he is "looking into it". From the newsgroup I can see I'm not the only one crossing my fingers.

If he doesn't then D2009 will be put on hold while I port 10 years of software to another database connectivity tool, so I can even open it in D2009. If he does, well then we'll be all set for moving onwards.

After I posted about having less screen real-estate, I found that using the push-pins gave me my entire screen. Nice touch. Those palettes now float into view if I need them, otherwise they float away. Very nice.

Thursday, 25 June 2009

The road to Delphi 2009 - Part 1

OK, so I started with the easy stuff. Installing third-party components.

Raize Components - Check
TMS Component Pack - Check
ReportBuilder Enterprise/Server - Check

OK, these were easy because they were designed for Delphi 2009.

ODBCExpress - Oh. Well, according to its lead developer Pieter Myburg on the support newsgroup, "we have stopped active development of ODBCExpress, and we won't be porting it to future versions of Delphi ourselves".

The last support was for Delphi 2006 which, as I understand, compiles fine in Delphi 2007 as that was a non-breaking release. But not Delphi 2009, oh no. So if anyone out there has any experience with bringing this component set up to Delphi 2009, I would love to hear from you. Without this, we're stumped. It'd be a case of going back through 10 years of software and rewriting it to use dbExpress or ADOExpress. And I seriously don't think we can afford that time.

By the way, bearing in mind I have come from Delphi 6 and have only seen screenshots of the IDE from Delphi 2005 onwards, I really like the IDE. Although I seem to have less code editing space because of all the other (really very useful) palettes that surround it.

Friday, 22 June 2007

Real-world tools #2: Database connectivity

OK, so yesterday I was on pretty sure footing with talking about ReportBuilder. Some of your comments and emails reflected that this is a popular third-party toolset and ranks up there with other report writing tools in Delphi. At least, undoubtely, most of you have at least heard of it. Today, I'm going to discuss a component set at the other end of the scale, although I may be proved wrong.

One of Delphi 1's initial selling point was that it was for the design of database applications, and was of course shipped with the BDE to give us connectivity to most of the big databases around at that time: Access, SQL Server, Oracle and Borland's own Interbase and Paradox. Eventually, like most of you probably, I looked for an alternative. Something that didn't require me to install the BDE for a start. I don't remember when or why this need occurred (can you?) but eventually I sought out a third-party solution. And the choice was plentiful.

At that time, our software was based around Microsoft Access (snigger) , and so although the BDE could cut it, we just didn't want our software to be the reason that the BDE was installed on the end user machine. In fact, the prerequisite for any alternative was that there should be no installation of any other proprietary files, period.

This led us to Titan Access by Reggatta Systems. This was ideal for us at that time, as it used the Jet engine (DAO) itself and only required a few files to be installed which, if they had Office, were on the PC anyway.

This was a great alternative to the BDE, and the components were so close to the BDE ones that it took no time to change the product to use it. This was fine until the company and the software grew out of Access and towards MS SQL Server. We knew the size of the companies we were trying to get into would have had SQL Server installed for their other Enterprise-level applications (at least this was the feedback we were getting) so it was natural for us to go down this route.

And although this is about components, not databases, I'm glad we did.

At this point, we had to seek another alternative. Reggatta also provide the same component set for Sybase SQL Anywhere but we spent some time evaluating this database and didn't get on with it for reasons I can no longer remember (this was mid-2001). So we needed a solution that enabled us to use both Access and SQL Server: ODBC.

At that time, the ODBC had a terrible reputation for being slow and clunky, but we seemed to catch it just as it was on the rise and approval for it was improving. Thank God we were right, it still remains a viable alternative and Microsoft continue to update their drivers for it every time they issue a major release of IE or Windows. There was talk some years ago about them starting to drop support for it in favour of the upcoming newboy, ADO. That never happened and the two now co-exist perfectly happily.

There were plenty of ODBC Delphi component sets on the market and again I cannot remember what it was that led me down the route to ODBCExpress. I think there was an article in Delphi Informant about it that probably swayed me, not least because then at least I had a tutorial in getting it up and running. Also, at that time, dbExpress had been launched but the SQL Server driver for it was "unavailable" and the way to use Access with it was simply to use an ODBC driver.

Since then, this component set has actually been one of the most reliable we have ever installed. I think we've only ever needed to upgrade to a new version once, and that was only because we were moving from Delphi 5 to Delphi 6.

Any weaknesses it has actually aren't in the components: they're in the drivers themselves. This is where you potentially have an issue. But the components are excellent, they do the job well and provide many of the options that ODBC supports. For those options that are database specific, it has the ability to tap into those as well so, for example, you can query the SQL Server ODBC driver about the database you are connected to, even though these calls are exclusive to that driver.

The strength of any ODBC component set is in the database drivers you are using. This is partly why we chose SQL Server 6 years ago, because we knew Microsoft would produce high quality drivers for it whereas we couldn't be sure of the quality of other database vendors.

The other beauty is that it has enabled us to toy with adding support for other databases (with a decent ODBC driver) to our software. This sounds good in theory but actually when you consider that none of them really stick to the same ANSI-92 SQL standards, you can end up with different SQL statements in your code depending on which database you're connecting to. Tim Anderson's blog had an article this week about SQLite and how they have had to deal with the same issue, so you wouldn't be alone in this!

The newsgroup support is adequate. It doesn't set the world on fire but Pieter Myburgh seems to single-handedly keep it alive and answer everyones' questions. But I do wonder how it would survive without him and you can tell when he is on holiday because the newsgroups don't get answered. Also, ODBCExpress used to be listed on Korbitec's website but it isn't any longer, even though Pieter's email address is still at Korbitec.

That said, the last release supported Delphi 2006 and it has a C++Builder version as well. There doesn't appear to be any movement for Delphi 2007 but given that it is a non-breaking release and the newsgroups suggest the 2006 version works fine under 2007, there's no need for them to bust out a new version.

This is a minor player in the toolset market and the database connectivity market doesn't seem to be as big as it used to be. Maybe because Borland released the dbExpress components and supported ADO out of the box with Delphi 6. And even more so when CodeGear announced DBX4 with Delphi 2007 for Win32.

I've thought about moving to one of those bundled Delphi offerings in order to save myself the worry should ODBCExpress go tilt. But I haven't for two reasons: firstly, I've used the same version of ODBCExpress for nearly 5 years without a problem - and that's without the source code (I know, what was I thinking?!); and secondly, because the database architectures offered with Delphi keep changing and I'm not sure I want to run the risk of changing everything over only to have it "deprecated" like the BDE.

What do you use? Why do you use it, why did you choose it and are you glad you did?

I'm going to be listening in one of the David I's CodeGear Delphi and C++Builder Roadmap Community Chat & Demos in a bit.

Wednesday, 20 June 2007

Tools of the real world

I should explain what it is I do with Delphi every day, which will maybe give a background as to why I've become what I consider a conservative developer. Conservative, that is, in terms of what software we use. The company I have worked for for the past 9 years develops three main products (payroll, HR and Time & Attendance, combined as a suite), written for Windows using Delphi.

Although we sell that product, maintain it and enhance it, we also write a lot of bespoke plug-ins and utilities to complement it, for the particular organisation that it is going into.

Given that very simple overview, I'll go on in coming posts to what third-party tools we are using, why we selected them, and maybe why we are still using them even though there might be a better or more up to date alternative.

I think the key point to make is that this is real life. This isn't the ideal world of (the fantastic) Coding Horror, these are real-world, business-oriented decisions that are made because we are under tremendous pressure to deliver, we have a 9 year old project and literally dozens of plug-ins and utilities that rely on consistency throughout.

It also doesn't help that this was only my second job which I started at the age of 23. It turns out I didn't know it all then like I thought I did, and God knows I don't know it all now. So there are some mistakes of youth that are still embedded in the product that occassionally we have to work around. The company have been very good in the past in allowing me an extra month or so to refactor some of the big ones out, even though the end result to the user is no change. Which in the long run is time well spent because future work can then be written a lot faster, neater and cleverer.

It may come as a shock now to learn that, as a base product, we are still using Delphi 6. I will come to why at another time, but it may explain why we're using some of the third-party component sets that we are. I will go through each of these individually over the coming posts, exalting them, damning them or just saying why on earth we're using that one instead of another one. They are: