Zac Bowling’s Blog

I just don’t know what to make of this video… I guess Microsoft is trying to break the stigma around Vista as best they can (I don’t think this video is helping their case much).

Texas Instruments, my company, will not support upgrading to Vista across the entire corporation as was leaked to the inquirer back in late 06 to early 07. No Vista machines in sight here still (except in the testing labs). Everyone is running XP if they are using Windows, but you do see a lot of Mac’s running Leopard, or and few laptops running the latest Fedora, Ubuntu, SuSE, and OpenSolaris desktops for those of using in various development and engineering roles.

I’m sure TI would probably use Vista if Windows 7 isn’t out by the time that XP goes into legacy support status, but XP works and we don’t have any problems with it. We already bought our huge corporate level volume licenses for XP so I can see the logic into why really IT doesn’t want to upgrade more then 10,000 users machines here when there isn’t a significant gain (and more of a loss really) by doing so even still. Nothing in Vista (even in SP1) is really worth upgrading for.

The developer program is turning out to be bunch of hype for something stupid. The restrictions on what your application is allowed to do is total, laugh-out-loud, crap.

I was initially excited about the SDK and developer program for the iPhone. I was willing to live with Apple being only distributor of Apps and getting a cut of the profits. I was willing to live with an entrance criteria to get into the App Store.

But then I heard about the other restrictions, and if you haven’t heard yet about the restrictions placed on your applications, here are gritty details:

• “Applications may only use Published APIs in the manner prescribed by Apple and must not use or call any unpublished or private APIs.” (section 3.3.1 in the SDK license agreement) That means currently no access to IOKit
• “An Application may write data on a device only to the Application’s designated container area, except as otherwise specified by Apple.” (section 3.3.4 in the SDK license agreement). No full file system access basically.
• Your application is completely closed when the user leaves it like when they get a call or any of a number of actions happen. (specified in the Human Interface Guidelines). Only one application can run at a time basically so no background applications. (Something you can do in the current Open SDK)
• “An Application may not itself install or launch other executable code by any means, including without limitation through the use of a plug-in architecture, calling other frameworks, other APIs or otherwise. No interpreted code may be downloaded and used in an Application except for code that is interpreted and run by Apple’s Published APIs and builtin interpreter(s).” (section 3.3.2 in the SDK Agreement)
• “If Your Application includes any FOSS, You agree to comply with all applicable FOSS licensing terms. You also agree not to use any FOSS in the development of Your Application in such a way that would cause the non-FOSS portions of the SDK to be subject
  to any FOSS licensing terms or obligations.” (section 3.3.14 in the SDK license agreement)
• No way to write plugins for the apps that exist on the phone right now, even for apps that openly support it plugins like Safari.
• Applications can be rejected for not meeting the Human Interface Guidelines
• No VoIP apps that work over cell network (that is acceptable, VoIP is a bandwidth hog and with unlimited data, it hurts the carrier)

Porting Firefox would be impossible because it supports downloading and running “interpreted code” with its Javascript engine. Things like Mono could only be used if you included a version Mono with your application, so basically no system wide install. No way to write a usable IM or IRC client on it, since apps can’t run in the background. Since we won’t be able to write plugins for existing apps like Safari, it means no Silverlight/Moonlight and no third party Flash support. IOKit isn’t in the list of document APIs even though its on the iPhone today, so no way to access the dock port to get to the PC or access the Bluetooth hardware. No way to tie into the data sync integration as well, so you can have it sync data with other things on your PC when docked.

What we get is so much more limited then what you can get natively in the unofficial Open SDK that is out there for the jail broke iPhones today. What annoys me is that Apple only played it off like their was only restrictions around application acceptance criteria for the application if was porn or a bandwidth hog. Not all this crap.

I think its idiotic of Apple to think this is what we were looking for. It’s pretty much a giant joke. I hope the public out cry about this is enough to knock Steve out of his black sweat-shirt and blue jeans, and have another wake up call.

Yesterday, I attended the first day of my company’s huge T³ conference. It was something else and a little different then the software conferences that I’m used to attending. Nearly 3000 people there. So many teachers from around the world. It made me appreciated the reach our software and hardware has a bit more getting to see how large our customer base is. As a software developer, I think we sometimes get separated from our customers through so many layers, and for me it was great to get to meet so many of them. One part thing I love about our company is that we try to get everyone in just about every role out there to interact and listen to our customers. Everyone from software developers, QA testers, project managers, distribution, sales, marketing, customer service, product managers, HR, configuration management, hardware/electrical engineers, and just about everyone else in our division came out to help with the conference.

I also met up with the Vernier guys there, one of the members of the GNOME Mobile community, and I got to play with their new GNOME based LabQuest. The Vernier guys are awesome. I’ve enjoyed working with them in the past to make sure that our TI-Nspire product works with their Vernier EasyTemp®/Go!® Temp/Go!® Motion probes and sensors, so they can power the data collection features of our product. The Vernier guys are big Linux supporters which is nice to see.

I’ve come across some really bad unit tests in a number of projects recently. I’m assuming that were written in the past only because someone said to the developers something like “Writing more unit tests improves code quality, so everyone write more unit tests!”. That was all the direction they got it seems (I’m assuming that from reading the code), and unfortunately, the developer and the leader wanting the developer to write unit tests wasn’t exactly sure why unit tests are important or how to write effective unit tests.

There are a few common goals that writing units help solve.

1. Unit tests provide an excellent facility for regression testing, in that help they allow programmers to easily refactor a piece code at a later date, and make sure that each module still works correctly.
2. Unit tests can be written to help simplify integration if you are working from the bottom-up in many cases. This goes well with really good planning around integration points. In a large project you may need to separate everything out into smaller units or modules. All the pieces are probably going to developed concurrently by different developers and teams. If you write unit tests around your integration interfaces, you can flush out any issues before hand and move on to develop other pieces of the project, and that lessens the strain on your integration phases later.
3. Unit tests can also offer a form living document for others to understand how the code works and how they can use it or call it by providing a bunch of working examples. Since the unit tests are constantly being updated as code gets refactored they are usually more reliable then any static documentation which can drift and get out of date.

Another overlooked piece is when should you write unit tests. I know that in a lot of projects that I’ve worked on, the time developers would take to write their unit tests is usually delegated to the end or when they have any extra time. The unit tests usually end up the victim of procrastination and a lack of coverage usually causes more headaches later.

I believe unit tests should almost drive your development, especially when you are developing in an agile environment, defining features as you go and you are constantly refactoring your code.

Unit tests are great way of showing you where you have spaghetti code and complex environmental dependencies right away.

If you enforce unit tests on all business logic, you pretty much can enforce a clean separation of interface and implementation, and drive you code towards using better coding standards and patterns out of necessity to just be able to test your own code (like moving your code to a basic model/view/controllers pattern). It’s a great practice to get into for writing maintainable code.

You have to be smart about your unit tests though. More is not always better. You have to be practical and not walk about until you feel you have sufficiently covered everything. Anything that preforms any custom logic based on a specific input is a big candidate for a unit test. Sanity check unit tests are not usually necessary (such as writing a unit test around a property/setter-getter method that simply reads or writes to a field in a class).

One of my pet peeves is a test that knows the internals of a particular unit, not the function of that unit provides. You should pretend you don’t know how your code works internally, and write tests how other code is going to interact with it. The exception to that rule is if you know of ways that unit could easily break with specific inputs but stay within the confines of what that unit provides you. Do not write units tests that simply reconfirm that your code works exactly the way that you wrote it and would probably break if someone refactored the internals of that code.

For example, lets say I have a function called “SaveCustomer(String id, Customer customer)” that I know saves to a database underneath and another function called “LoadCustomer(String id)” that loads from that database underneath (the underlying database is not intended to be accessed directly except by the customer abstraction in this case). It would not be an effective test to write a test that calls the SaveCustomer function and then connects to the database to see if its written, or similarly, write a test that calls the LoadCustomer function and checks to see if the data returned is the same as in the database. Rather a more effective test would be write a test creates a customer and attempts to store it using the SaveCustomer function and then tries to load it back out using the LoadCustomer function and verify the data in the customer objects survived round trip.

Another example that I ran into was a function called “getOffset” which was meant to return the internal position of where to read off an array returned by that same class. The previous programmer that wrote the function knew that the value would always return 16 at that time but made the function so that he didn’t have any magic numbers running his code and that he could change the value it returned to something different someday (or possibly even dynamically figuring its position at some point). However the programmer wrote a unit test to make sure it always returned 16 and sure enough when I went in and changed what the offset would was, the unit tests broke although no code did that called that function. Effectively, the test was broke and not the code.

It comes down to being smart and practical about your unit tests. There is no perfect setup. I try to shoot for a test around each public function and protected function (if intended to be inherited by other classes), and if applicable how the class is used as a whole (thinking of it as almost writing examples and documentation for the code).

Some good candidates for unit tests are around that code that does logic in loops, contains switch statements (especially in languages that all statements to can fall through), any goto/jump statements, any regular expressions usage, any “auto-magical” features that can possibly fail at runtime, non generic collection usage, null pointers, static position access on non-rigged arrays, primitives based by reference, and code that calls reflection features of your language.

In scripting/interpreted/dynamically typed languages, lots of sanity check tests are also more useful to attempt to test each code path possible. I wish you luck if you are writing a large application in one of these languages if you don’t have a strict coding standard and a whole bunch of tests.

Whatever you do, you can’t trust unit tests to be your only type of testing. Unit tests are not well suited for GUI testing and they can almost never do functional testing. Unit tests can only tell you if an error occurs in one particular case but not that couldn’t foresee to happen in many cases. Automated testing solutions depending on your project are really important as well in many cases.

What the hell is the world coming too. I found this in my local grocery store tonight. $1.50 + credit card interest is totally worth the convenience of not having to handle a dollar bill.

Update -
Found the company’s product docs. “Bright, flashing blue LEDs to attract customers.”

This is the best thing that I have seen all day! http://www.conservapedia.com/Special:Statistics. Not gay at all.

I blogged earlier about getting Phalanger running on Mono and the missing feature missing being the native extension support because the bridge was written in Managed C++.

Well I came up with an all C# solution. It’s not close to being done, more just a proof concept. I’m not sure the willingness of the upstream with supporting this method because it probably means a lot more work to maintain this method then the managed C++ version. With the managed C++ version, its less complicated because at least you can reuse the headers from PHP to help out with all the function declarations, typedefs, macros, and structs (that sometimes break ABI compatibility on rare occasion in PHP from version to version which everyone who has ever used the Zend optimizer knows about when they upgrade their PHP version the old version of Zend’s optimizer breaks). With the all C# solution, it means reversing all those headers and duplicating their function in C#.

There are also a few typedefs in the headers that compile to two different sizes between 32bit and 64bit platforms so I have to declare two versions of the same p/invoke calls on some occasions and use one or the other if the code is running on 32bit or 64bit.

The way the code works on the managed C++ side is that the code will iterate through all the php extension dynamic libraries. All the extensions in PHP are required to expose a standardized set of function exports. The managed C++ code will dlopen them (I’m using Unix terminology here, LoadLibrary on windows), use dlsym to get the function pointers for that extension, store them in to a struct, and stick the struct on to an array where it holds references to all the extensions. (I’m way over simplifying what it does, as phalanger also does some neat isolation tricks by loading the extensions into a different memory spaces/processes to keep buggy extensions from crashing the process phalanger is running in and remotes into them).

This same kind of operation using dlopen and dlsym like this can be done from Mono/.NET natively in a cross platform safe manner using just plain old P/Invoke and some really cute System.Reflection.Emit code. (Cecil maybe able to help me here for what I’m doing later). Basically, it means I emit an class for each extension on the native side, with all the emited classes inheriting from a interface that matches the same method signatures of the exports I’m P/invoking in on with each lib. It’s not possible to declare an extern in an interface, so I emit both a private pinvoke call and a public function with the same signature and name that just calls it which is in the interface. The class I emit each time is almost identical except for the given library name each of the p/invoke calls call out to. Poor mans dynamic p/invoke. This entire process can be at runtime, or done before hand, where I can generate assemblies or even exes that remote into a parent process for poor mans process isolation (which is really easy since I’m using a well known interface for each of the extensions so remoting to it is simple).

After the extension is loaded up, the managed C++ code will then do some System.Reflection.Emit operations to generate a wrapper assembly around the functions provided by the php extension library. This is done using the reflection like information provided by the extension to emit a similar call in the .NET side that proxies down to the extension. PHP has setter and getter methods, overloading, purely dynamic functions, well known functions, etc. Thankfully this managed C++ code converts to C# without to much trouble. There is still a lot of structs to rebuild though.

In all, its about a 6 to 8 week project if I had the time. (which I don’t)

In my free time, I’ve been trying to get Phalanger working better on top of Mono. Phalanger is a PHP implementation for the CLI and ASP.NET. It runs like a champ with the basic features on top of Mono.

One of the greatest features, and also biggest missing feature with running Phalanger on Mono, is native support for PHP extensions. The extensions in PHP are basic dynamic libraries, exposing a common set of standard C exports. In Phalanager, they have written a wonderful bridge between them, but unfortunately for me though, this bridge is written in Managed C++ (icky) and does a lot of Win32 specific things. They are also doing a lot with named pipes and running things in separate memory spaces, and worst of all, a good portion of the managed C++ code is doing Reflection.Emit. No easy way to get around this it looks like. *sigh*

Apple seems to have given up with supporting Java on the desktop. They have really been pulling back hard on any desktop Java related technology.

It’s only on the desktop. Apple’s J2EE based WebObjects technology doesn’t look like its going anywhere for now (Apple’s store and .Mac server are still powered by it) but that is a server side technology that was created back in Job’s NeXT days which is included in Mac OS X Server.

Apple slipped and didn’t ship Java 6 with Leopard like they implied they would. They also pulled the developer previews of Java 6 from the Apple developer site. What is left for Java 5 support in Leopard is sadly broken around everything AWT and Swing related. Not that I’m horribly sad to see it going but years ago, Apple was excited about supporting Java. I remember claims that Mac OS X would become the platform for Java on the Desktop a few years ago at an Apple Developer Conference.

About a year and a half ago, Apple still supported Java bindings to Cocoa and Java was treated as a first class citizen in Xcode for developing Cocoa apps. That was depreciated, and Apple’s solution was to tell people to migrate to Swing instead and publish articles on the ADC on how to embed the JRE and use JNI with Cocoa and Objective-C code. Long ago, Apple had bindings for Quicktime that fell into depreciation in Jaguar. They also had bindings for other libraries but they are long since gone. Before Leopard, there was almost as many references to Java on the Xcode and OSX pages as there was to Objective-C. Now you can’t find anything. Old links all redirect to developer.apple.com/java.

You also don’t see Java on the iPhone, which isn’t a huge surprise. Jobs was quoted as saying right before the iPhone’s release, “Java’s not worth building in. Nobody uses Java anymore. It’s this big heavyweight ball and chain.”

There has been a lot of talk about porting OpenJDK to the Mac, but I don’t really believe its worth the effort to take on that beast. Why not instead take the opportunity to embrace something already supported on Mac like Mono (or maybe Python) ?

It’s a sad fact of life.

1. You can have it Fast and Cheap but it won’t be Good.

2. You can have it Fast and Good but it won’t be Cheap.

3. You can have it Good and Cheap but it won’t be Fast.

It also works for project schedules, resources, and scope.

1. The project can be done with that many people and that deadline if you allow us to change the scope.

2. The project can be done with that many people and that scope if you allow us to change the deadline.

3. The project can be done with that scope and that deadline if you allow us to change how many people.

After much debate, the OSI has given the stamp approval on both the Microsoft Public License (formerly the Microsoft Permissive License) and the Microsoft Reciprocal License (formerly the Microsoft Community License) as both Open Source licenses.

Friday, 12 October 2007

Acting on the advice of the License Approval Chair, the OSI Board today approved the Microsoft Public License (Ms-PL) and the Microsoft Reciprocal License (Ms-RL). The decision to approve was informed by the overwhelming (though not unanimous) consensus from the open source community that these licenses satisfied the 10 criteria of the Open Source definition, and should therefore be approved.

The formal evaluation of these licenses began in August and the discussion of these licenses was vigourous and thorough. The community raised questions that Microsoft (and others) answered; they raised issues that, when germane to the licenses in question, Microsoft addressed. Microsoft came to the OSI and submitted their licenses according to the published policies and procedures that dozens of other parties have followed over the years. Microsoft didn’t ask for special treatment, and didn’t receive any. In spite of recent negative interactions between Microsoft and the open source community, the spirit of the dialog was constructive and we hope that carries forward to a constructive outcome as well.

The Open Source Initiative is best known as the steward of the Open Source Definition and for its license review process. But, an open source license is just the starting point. Open source depends upon code (which can be made better), community (which can be made larger), and ultimately a commitment to the idea that the more free the market is for innovation, the more innovation the market can deliver.

Every approval that OSI issues represents our community’s demand for more open-source code, a larger and more vital open-source community, and all the benefits open source brings to innovation in a free market. The new Ms-PL and Ms-RL are no exceptions.

(source http://opensource.org/node/207)

There a lot of new standards and formats coming out out there that require an implementation of Java to be compliant I’ve noticed lately, and its seems to be growing. I just can’t figure it out. Is there nothing better for what they are trying to achieve?

For example, Blu-ray requires an implementation of Java on all Blu-ray players to run the interactive menus on those systems. The Blu-ray menus are stored in Java byte code and the player’s embedded Java runtime runs this content. I think this is partly might be why Microsoft is so adamant with pushing HD-DVD over Blu-ray (HD-DVD’s menu system is stored in a standardized document format). If Microsoft ever wanted to support Blu-ray in the XBox 360 or in Windows Media Player it would probably require embedding some form of Java (which would happen the next time there is a cold day in hell) or by converting Java code to .NET like IKVM.NET or their Java to J# class converter. It pretty much isn’t going to happen which is a bad thing because I believe Blu-ray is better then HD-DVD in almost everyway (except for requiring Java).

Java is also a requirement on the new OpenCable based Cable Card system (this is CableCard 2.0 basically) as part of the new OpenCable Application Platform (OCAP). It requires anyone that has a OCAP compatible TV or PVR to host an implementation of Java to display the content. This is so your cable provider can send down their own menu system in Java byte code to display on your TiVOs and new CableCard ready TVs (not something I’m looking forward to since right now they show banner ads on 1/4 of the screen with their on screen menu guide on my current cable box) and how to change the channel and send back data in their own protocols to order things like on demand services and PPV since all the cable providers can’t seem to decide on a common two way protocol for these services. What sucks is that cable providers need something that lets them control how you request and change the channel if they want to add any more channels, convert more to HD, or have more on demand content (google for switched cable service) and I hope they don’t require OCAP to do that and the FCC steps in to prevent it.

What annoys me is that our government, specifically the FCC, looks to CableLabs (the company who came up with this OCAP standard) for the standards that they want to enforce the all the cable companies and television manufactures to use. The FCC enforced the first CableCard standard, and if they enforce this one, the FCC will be enforcing Java on everyone. It would be good to mention that Microsoft is working in this same field as well with the partnerships with Comcast for their television delivery platform, and also AT&T and Verizon (FiOS) to work with them on their IP based TV services on their own proprietary system.

Java is also the only development platform on a good number of cell phones out there also (which probably isn’t news to anyone). J2ME’s dominance isn’t that big of a deal I guess since Symbian, Windows Mobile, BREW, Palm, Blackberry, and others provide us with various alternatives which are usually a lot better then using Java. Java probably will continue to rule as the platform of choice on all the cheap-o phones.

Java is now even influencing the design of all the new ARM chips out there. Jazelle extensions are pretty much required to be ARM compliant now which is great for speeding up those slow J2ME apps by running Java byte code (with some slight assistance) at almost an instruction level. It was created out of need since J2ME is horribly slow on most cell phones and usually requires long compile times when first loaded. Jazelle makes me cringe because we can’t get the docs on how these new op codes work to use for anything else except for Java, which could possibly help someone speed up something like .NET, Mono, LLVM, or any other JITed platform when running on ARM.

This all has to make Sun so happy. Heck, they even renamed their stock symbol to “JAVA”. But still why Java?

For Blu-ray and OCAP, the companies designing the content want something that has lots of flexibility (a programming language and platform provide them that but a good standard would probably be better in the long run). The companies building the hardware for the players, cable boxes, and TVs want a format that is secure and can run in a protected environment like a VM that they can be tightly controlled, and is portable between any current or future hardware.

For TV and Video services, Java fits the requirements I guess, but it seems like there are so many other and possibly better alternatives. If you want a platform that you can control, that runs in a VM, just off the top of my head, you’ve got Python, LLVM (for all the GCC languages), Mono/.NET, and bunch of others (not to mention you could roll your own basic scripting like language or custom binary format as part of the standard). It seems to make sense more then trying to provide a development platform, to create set of non propitiatory standards (possibly based on XML to keep it simple) to handle these types of services and handle almost every situation that you need. On top of that, you could possibly provide your own development platform that uses that data but it’s use remain strictly optional for the end user. For the current uses of JavaTV and JMF (Java multimedia framework) in the OCAP and Blu-ray standards, it would probably benefit the consumers so much more in the end if they did that.

Honestly, I believe Java has a lot of inherit issues that pretty much can’t get fixed without breaking most backward compatibility (to many things to cite here), which is why I contribute to and support Mono and .NET. For the record though, I don’t hate Java though, and I spend a good amount of time at work developing in it. It just seems like there are so many better alternatives.

One of the only things good about it all is that at least Java isn’t entirely proprietary anymore (although good many of the common libraries, frameworks, and implementations remain that way).

I’ve been trolling and occasionally commenting in the on going discussions on the OSI license approval process since the beginning with the approval of the Microsoft Permissive License and Microsoft Community License still on going. Microsoft submitted for both licenses to become OSI “Open Source™” approved licenses. Microsoft has done a wonderful job listening to the community, and really has gotten involved in the current discussions over the licenses.

The news of Microsoft doing this making news on Slashdot and Groklaw, it was almost certain to be a prime piece of flame bait. However the anti-Microsoft zealots who have been getting the mailing list info from from Slashdot and Groklaw are all being ignored and usually are silenced and suppressed, while the real honest discussion and debate about the license and not the personal feelings some have about the company who submitted it continues on.

One of the debates was over naming of the licenses. There was confusion by some over the titles of the current licenses. The Microsoft representatives and lawyers quickly offered up alternatives to see what we think and later resubmitted under two new names.

The Microsoft Permissive License will become the Microsoft Open License

The Microsoft Community License will become the Microsoft Reciprocal License.

While I don’t personally think the old names were confusing, I still applaud Microsoft for their effort here.

Cheers to everyone at Microsoft that helped make this possible!

I got some interesting emails earlier this month on my OSI mailing list, but it doesn’t seem to have garnished much attention out there. Microsoft has been quietly doing something positive for us in the F/OSS world.

Jon Rosenberg of Microsoft’s Open Source Lab, has summited both the Microsoft Permissive License (Ms-PL) and Microsoft Community License (Ms-CL) to the Open Source Initiative (OSI) for approval as “Open Source™” approved licenses.

These licenses do meet the criteria of both the Open Source definition and the Debian Free Software Guidelines. Having the OSI approve these licenses does a bunch for us developers in the F/OSS community in that it allows the public to comment on the issues and problems in those licenses. All of use developers that only work on this in our free time can’t afford to hire lawyers to review the issues with every single license we come across, so the comfort of having the approval of the OSI group helps.

While I’m so happy Microsoft is doing this, I do have a few small reservations. In general, I don’t think having more open source licenses are a “good thing™” for the community. I especially don’t care for any new licenses that overly govern or even disallow mixing code under one license with code under it’s license. We have problems with that type of thing right now where FSF’s GPLv2, Sun’s CDDL, and all the Microsoft shared source licenses prevent mixing code with anything under a different license. GPLv3 did take care of part of this issue to be compatible with the Apache license and a few others.

However, the problem is not only with mixing code across different licenses but also different versions of the same license. We are running into problems where we are not able to mix code that is released only under GPLv2 with code under GPLv3 code right now and its also causing confusion mixing LGPLv2 with GPLv3 code. One of the problems is that if Microsoft releases a new version of their license, any new code under the new license can’t be mixed with code released under the older versions of the license.

I have a few suggestions. Since Microsoft is releasing multiple licenses, why not have a clause to allow us to relicence code under MS-PL up to MS-CL (as in LGPL to GPL) so we can mix some Ms-PL code with code released under the Ms-CL in the future? Right now thats not even possible. The Ms-PL license also gives you a lot of freedoms and on face value seems similar to the intent of the MIT/BSD license except for the issue of mixing code with other licenses (MIT and the new BSD licenses don’t say pretty much anything more then you have a license to do anything you want with it except claim you wrote it and that you get no warranty). If you are planning on using Ms-PL code in your application and you plan to only ever release in binary form only, you can have almost every right you have with the MIT/BSD licenses, but if you ever release in source form, all of your code has to be under the Ms-PL.

All in all, this is positive thing. Awesome step forward Microsoft.

DVD Jon, the same guy that brought you the wonderful but short lived cross platform iTunes store client and figured out the method of encrpytion on Apple’s DRM format, did it again.. You can now activate your iPhone without service from AT&T to use it as just an iPod and to surf the web over Wifi. The awesome part is that his code works on Mono which opens the door for anyone running on Mac (using the Windows version in Parallels or VMware at the moment) or Linux (by way of Crossover/Wine/vmware).

Get the binary here and the source here.

For Mac and Linux users, make sure you kill any webservers or anything that might be listening on port 80, then simply add “127.0.0.1 albert.apple.com” to your /etc/hosts, unzip the binary file above, and run “sudo mono PhoneActSrv.exe” from the console. The rest of the information is the same as on DVD Jons site.

NOTE: no body has hexed the itunes for mac yet, but I’m sure it won’t be long. It will work in a Windows versions of iTunes in Parellels or VMWare.

Seeing how I bought an iPhone and I’m paying for the service, I have don’t have a use myself for this but I’m sure someone out there will find this useful.

Happy 4th of July!

When I read Andreia Gaita’s latest post, I started to tear up and laugh a little as it brought back a lot of painful memories with my experience with fighting it. Andreia, you know all to well my pains and my struggles I had with that code (trying to embed Mozilla externally). I’m so overwhelmed with joy that you got it going. Awesome work that you have done taking my pretty hobbled and badly broken code the rest of the way and making it actually work!

My biggest hurdle in the beginning was trying to figure out how to make this all work given all the requirements. Some of the requirements where more self imposed in what I though it would take to get the web browser control accepted, and others requirements came from other monoers (especially Miguel ). Going through the options, it came came down to forking off a version of the entire mozilla tree and embedding the entire mozilla engine into my wrapper, or calling it Mozilla dynamically. (You can read more about that in previous post). I knew that forking the mozilla tree would be horrible because no body would want to build it and use it (it takes an extra 7mb, about 3 hours to build on most average machines, and you have to manually maintain it). Also there were concerns about littering the source control with that massive beast. That left the only other solution, externally calling the local Mozilla environment on the users machine dynamically, although it was the FAR more complicated solution to make it work (it shouldn’t be and perhaps this project will help that).

In beginning, I was still going through the mozilla code and documentation trying to understand it better. I have written a bunch of patches for GtkMozEmbed, so thats the angle I started attacking it. The Mozilla documentation on embedding is helpful but it was written around the idea of forking the Mozilla tree. I was still trying to understand how to make it work so I went ahead and wrote a version of the embedding control in that fashion (internal wrapper) by forking the gtkembedmoz code and removing almost all the GTK+ references (you can see older bits and peices of that here). I though I could take that version and move it to build externally, however I didn’t realize how many private functions and interfaces are inside Mozilla that gtkmozembed actually used. There was almost no way it was going to work, so I started from scratch.

I was pretty much forced to dive deep into internals of XPCOM. In the process I had to learn the evolution of the Mozilla codebase to understand why things where done the way they are done. This took way to long (I could almost write a small book on the topic now). When I got done, I figured that no body in their right mind (I guess that includes me) is going to want to go through this to just embed Mozilla. Thats when I came up with the goal of making sure that the backend library that I was going to write to bridge Mono and Mozilla would be reusable outside of just Mono if at all possible (a simple embedding interface that exposes basic C exports for people to use).

Then it came down to the build environment. I have experience with witting libraries in the open source/Linux world, and at first I though it was going to be easy. Heck, even my distro even had package config files to save me a bunch of trouble (later I found out that my distro added those and there is no standard package config for developer packages or XPCOM). I got started with an environment that I though would work for everyone and got going writing code. I got it working for while, but half way through development I updated to a new release of my distro. Suddenly nothing would compile anymore. I found out that my distro was including IDL files it wasn’t supposed to in the older version. I was missing IDLs I needed. I did some investigation of other distro setups and found everyone did it differently (vastly different in many cases). This shut any new code development down while I re-evaluated what to do and what code would I be able to saved. I went back and looked at what was frozen and I trimmed down the interfaces I was going to use to as few as possible. After that I went back to the code and started trimming out code that called those interfaces I couldn’t/shouldn’t use anymore and looked for a work around to each one. This put the code in an unstable state where it wouldn’t build again. I hacked and hacked, trying to get it to compile again. I wasn’t even sure when I was done if it was going to work since I didn’t get to test it as I went along. I spent a good month’s worth of free time on it. I finally got it to compile but I was cheating in a few places. My build environment still needed work to stop cheating. I remember only one time that I seen that come up and show something after that after hacking it for a while.

Shortly afterward, I started working for TI and at the same time my laptop decided to take a sabbatical and go off to laptop hospital at Dell, who after replacing my motherboard, happened to wipe my drive and image with Windows (thank you every much Dell! Of course it “wouldn’t boot into Windows”, its called LINUX! Thank god I backed everything up.) I decided to give up working on it and wait until the interfaces changes and patches I submitted to mozilla would make it into the next release version (basically by 2.0).

I haven’t been able to spend the time to get back into it with work taking more of day. I did take some time and finished some work on getting a reusable build environment last month but haven’t been able to dedicate the time to it I wanted to.

Thats where Andreia has stepped up and taken over getting it done and she has done an awesome job doing so. I’m so happy to see it going somewhere with all that time I dumped on it.

I went and did it.. I waited in line 2 hours and got the last iPhone at the local Cingular/AT&T store. Over 200 people didn’t get one.

The line a was long too. It was insane. The cops asked a few people to leave that got a little crazy when they anounced they were out. Way to crazy… You can see a few picks on my flickr and possibly later tonight when I get it activated.

First off, I have to praise “Shana” (Andreia Gaita) for working on my old Gecko based WebBrowser control for the 2.0 version of System.Windows.Forms. I had ideal goals, lots of hopes, and not enough time to spend working on it. I have a job and some what of a life, which that only leaves a few hours each day these days to dedicate to Mono (a common cry of many of us open source developers that are not privileged to be employed to work on this stuff full time).

The Mozilla guys did a really good job to make Gecko reusable, even though its not their primary goal. Still, there are still a lot of problems. Some of them are legacy issues that steam all the way back to the way XPCOM was designed and how old Netscape ran things (releasing binary SDKs to work with Netscape with only the static and public interfaces they were willing to support exposed). Since that time, the Mozilla products have been in a sort of flux with how things are done in the open source world and how Netscape used to do it with respect to how each product is built, packaged, and released on each operating system and distribution.

Netscape’s requirements were that they needed to be able to allow people to only specific components they wanted to be exposed and the members of those classes they wanted to be visible, and they needed a form of reflection and an interface system that could help people access and expose their classes from and to other languages like from external C++ libraries, Java, and Javascript (and then later Python, Ruby, and Perl). They were influenced by the design of COM (as was Ximian when they designed Bonobo), and decided to write an IDL compiler (like MSIDL) and even reused the libIDL used by Bonobo, and that is where the XPCOM standard came from. Rather then write two sets of header files (one public and one private), all the interfaces would have a IDL file for them describing their classes and Netscape would only release the ones that were frozen and stable in the SDK. In a perfect world this would be all you would need to dynamically call into XPCOM; unfortunately, the SDK has not been the primary focus of the Mozilla project and many of the needed interfaces to do anything useful for the past 7 years, have not been frozen, making it the SDK almost useless.

In the past, the only (supported?) way to embed Gecko was to fork the entire Mozilla tree and change their build setup to fit your needs and statically compile your own implementations into your own setup. This is far from being a very elegant solution. You end up taking on the entire Mozilla tree (thats 20 to 30mb of code IIRC), and Mozilla takes a very long time to compile (a few hours is not uncommon, which just about every Gentoo user knows that pretty well). The best way to handle it would be to dynamically link an existing Mozilla installation.

Then along came the concept of the GRE (or Gecko Runtime Environment). This idea was to use a common GRE runtime for all the Mozilla products that would be guaranty binary compatibleness between each major version. This idea never really worked to well (mostly because the SDK was useless) and was phased out in later Mozilla, Firefox, and Seamonkey releases.

A number of embedding libraries and solutions started popping up in the Mozilla tree, but all of them still required you to build the entire Mozilla tree just to build the wrapper or library for your language or toolkit to embed Mozilla. One successful solution in this arena was GtkMozEmbed. It is built by default in all Linux builds along side whichever product you where building, allowing you to embed Gecko by dynamically linking libgtkmozembed.so that was pretty much included in every released Mozilla project. This exploded and allowed projects like Galeon and Epiphany to be born. However GtkMozEmbed is very limited in its functionality and only works for GTK+ users. We also can’t simply fork GtkMozEmbed because it calls code that is only available internally in Mozilla, and so its not possible to use if you are building externally.

The only solution left is to go against the rule books and use “unfrozen” interfaces. It works but the only problem now is providing a solution that allows everyone to build the library in their setup against the SDK or using the funny developer packages in each distro (there is no official way to build a developer package for Mozilla products except for building the SDK, so just about each developer package is vastly different from the others).

Thats what the mozembed code does. It works by dynamically linking a XULRunner or Firefox setup you have installed. We build it against IDLs and headers we get from the Gecko SDK. The only downside is that since we are not using all “frozen” IDLs, there is a small chance the API could change in future versions, but I’m not really worried because it hardly ever does change and its easy to fix internally (its nothing like trying to maintain a module for the Linux kernel from version to version). What we do is add the missing IDL files that we need (taken from a later release of Mozilla that I built locally) to the IDLs that are included in the latest Gecko SDK and then generate header files from them to build against (You can see those extra “unfrozen” IDL files here in the SVN).

I love Mozilla and Firefox, but I understand why Apple choose to port KHTML and create webkit to build Safari, even when after they hired people from the Mozilla project to work on it. I’m curious to see what Alp and company do with the Cairo port of Webkit. Apperently the Apple release of Safari for Windows (by porting a few of Apple’s Core techonologies) set the Cairo port back a bit. It would be great if Apple would release all the Core* stuff as open source but I don’t think that is going to happen.

Omar snapped this photo of all of us Mono guys (Joseph Hill, “Wasabi”, and me) at the Dallas Code Camp this year. It turned out to be a really good showing. Can’t wait till next year!