Marko Grönroos

JPA and Vaadin JPAContainer

2012-01-21T00:34:00.000-08:00

I've been finalizing the completely rewritten chapter about Vaadin JPAContainer this week. I still need to check that everything really important is included.

I don't have almost any previous experience with JPA, which is kind of good thing when you write documentation about something. That way, you personally know all the questions a beginner would have.

JPA and ORM in general seem pretty important technologies. Whatever your application is, you most probably need some data storage. Using ordinary files is possible, but not near as handy as proper databases. However, stuffing complex class structures through the square holes of database tables was a hell before the ORM technologies emerged. XML databases and NoSQL were much nicer for structured storage.

Some years ago I did some experiments with related technologies, such as JAXB and Hibernate, but using them was a bit too much effort at that time. Using Hibernate with Vaadin really required a Container implementation, but there wasn't one at that time. Well, JPA is quite close to Hibernate, and Hibernate is currently one of the JPA implementations.

Vaadin JPAContainer makes things really easy.

// Create a persistent person container
JPAContainer<Person> persons =
    JPAContainerFactory.make(Person.class, "book-examples");

// You can add entities to the container as well
persons.addEntity(new Person("Marie-Louise Meilleur", 117));

// Bind it to a component
Table personTable = new Table("The Persistent People", persons);
personTable.setVisibleColumns(new String[]{"id","name","age"});
layout.addComponent(personTable);

So, just one command to get a container bound to data in a database table. JPAContainer is, however, just for binding user interface components directly to data. You can use it to add or edit the data, but using pure JPA is much nicer, easier, and more flexible for that.

EntityManager em = JPAContainerFactory.
    createEntityManagerForPersistenceUnit("book-examples");
em.getTransaction().begin();
em.createQuery("DELETE FROM Person p").executeUpdate();
em.persist(new Person("Jeanne Calment", 122));
em.persist(new Person("Sarah Knauss", 119));
em.persist(new Person("Lucy Hannah", 117));
em.getTransaction().commit();

Well ok, here we got the entity manager from the JPAContainerFactory, which makes that a bit easier than normally.

JPAContainer is a commercial product. That shouldn't be a problem if you're coding for a company that can pay the license. For free software, you can use the AGPL license. I've always felt that this sort of dual-licensing scheme is most difficult for "potential startups" where you experiment with a business idea, but do not yet have a company to back it up financially. You do not normally have the money to invest in licenses at that point, and a company might not want to invest much in such experiments either.

There are of course alternatives for JPAContainer. Using pure JPA is quite OK approach as well, combined with BeanItemContainer in Vaadin. There's also SQLContainer. It is also something I don't know about, as I didn't write the chapter myself. I would need to review and reorganize it in next weeks though, and possibly rewrite at some point.

Well, I did make a couple of mistakes while writing the JPAContainer chapter. The first one was trying to just update a really complex design documentation written by someone else, in a topic I knew nothing about. Writing is always a learning experience and it's not sensible to try to write about something very complex that you don't have a clue about. Sometimes, it's best to start from scratch. Unfortunately, I didn't do that when I started, but just over a week before the release.

Another issue is learning by doing. I usually start writing documentation by experimenting how it works, making some example code. This time, as I was in a hurry, I tried to skip that phase. I only started it as the last effort when I was totally stuck, some 4 weeks of struggle behind and 2 work days before the release date. Just in those couple of days, all pieces fell into place and I finally understood how everything works.

Wikipedia and Antipiracy Laws

2012-01-18T13:40:00.000-08:00

So, if you have noticed, we've had a Wikipedia blackout. I normally use Wikipedia to look up dozens of articles daily, so I'd be pissed if their protest wasn't really valid.

As innocent they sound, the "Intellectual Property" laws are a big threat, really. They are somehow understandable, which is the reason why they exist, but at the same time, they may be a big threat to modern civilization.

You might think that they are not a problem to you, as they only prevent criminals from making crimes. You might even think that you want your intellectual properties be protected.

Their aim is to prevent illegal copying. Artists and others work so that they would get paid, and illegal copying is kind-of stealing from them. So, forbidding illegal copying is rather understandable.

So what's the fuzz about?

Let's consider the problem with DVDs. They use the "CSS" encryption mechanism to prevent unauthorized copying. Fine, excellent, evil pirates suffer, right?

Unfortunately, watching CSS-encrypted movies requires special software that holds a special decryption key. The software is usually free-of-charge, but it's only for Windows, and perhaps for Mac as well. For Linux, which I use, there is no such authorized software for watching DVDs.

Fortunately, some Norwegian guy cracked the CSS key and wrote a decryption algorithm some 10 years ago and now there's "DeCSS", an algorithm to play (and copy) any DVDs.

So, over the years I've bought some 200 DVD movies and I've always watched them with Linux and nothing else. I've never copied a DVD that I've bought, only watched them.

However, using the DeCSS algorithm for any purpose is illegal in Finland. It's been tried in court.

The fact is that, once cracked, CSS became totally obsolete forever. The laws will never have any effect on unauthorized copying. But they will continue to prevent morally right use of your property.

So, pirates who operate illegally of course use the DeCSS decryption to make one copy which they then copy to millions. But people like myself, who buy DVD movies legally, are criminals. Trying to stop a few criminals, they incriminate perhaps millions of Linux-users around the world.

So why should you care? Linux users are somewhat rare, and you're probably not one of them (except if you use any of the 60% of the web or 55% of smartphones or a TV).

Oh, but it also affects Windows users. There's free DVD watching software for Windows as well, such as VLC, which is actually quite popular as it's very good. But it's illegal, as it has to use DeCSS. The laws prevent writing and distributing any free software that you could use to watch DVDs.

Obviously, preventing the use of free software to watch legal DVDs was not the purpose of the laws. It's just a consequence.

It's a good example of a case where somewhat understandable reasons lead to absurd results. Even so, the vast majority of the Finnish parliament supported the law. And there is a lot more IP legistlation on the way.

Oh, did I mention that Finnish consumers can legally copy movies and music for their own use and to friends. We even pay a tax for that when we buy DVD-Rs. But copying by circumventing CSS is illegal.

Given the absurd criminalization and heavy anti-"piracy" lobbying to take away your rights, you should have no sympathy for the movie industry. O'hoy pirates! Go and board them!

So, to SOPA or PIPA...

I'm not personally familiar with the new legistlation initiatives in the USA. As far as I've understood, it's something similar as the recent Finnish laws that allow blocking child porn and peer-to-peer download sites. The laws already have a sad history of abuse. For example, the laws do not allow blocking websites inside Finland, but they've nevertheless done so as there's no possibility to legally challenge the decisions.

The pattern is that "reasonable requirements" can have devastating effects. The current laws are largely based on the assumption that you are responsible for your own publications. The suggested laws place the responsibility on the owners of web servers and internet providers. So, if me or you were to publish the DeCSS code here in the blog, the Blogspot would be considered the "publisher" that is responsible for our writings. To protect itself, Blogspot would need to have editorial review for everything any one of the millions of bloggers write.

This would quickly kill almost all blogs, discussion forums, and websites in the Internet. Also Youtube and such. It would kill the basic freedoms that we take for granted, and most of the Internet.

Part of the Finnish goverment already has plans to expand the blocking further. Some want to block information about drugs and weapons. Then racism of course, blasphemy, information about bombs, porn, denying the Holocaust, undemocratic thought, ... anything to protect the children, leaders, gods, and corporate owners.

Even though basic IP laws may be necessary, the recent development upholds corporate ownership rights over consumer ownership rights. They are used to buttress dentrimental monopolies and force you to pay vastly overcharged prices, even for basic necessities. They distinguish diversity and freedom, which are the life forces of the modern technological and scientific ecosystem.

So when you see the blackout screen at Wikipedia, just imagine the day when it could be blacked out by your government.

Vaadin IPC for Liferay add-on

2012-01-03T14:08:00.001-08:00

For Vaadin users, I just finished writing documentation for the Vaadin IPC for Liferay add-on yesterday. The version 1.0 of the add-on will be released tomorrow I think. The add-on is very very useful if you want to have two portlets that communicate with each other on the same page. The add-on uses standard Liferay JavaScript event API, so the other portlets can be any Ajax portlets, not just Vaadin.

Here's how it works:

As a side note, I really like Inkscape for drawing those diagrams. Well, I have no clue how wonderful some "best-of-the-best" vector graphics apps are. But I seem to be able to do practically everything with Inkscape and, after a while, it's not so hard to use. I've also done a nice production pipeline so that I can just save the pictures as SVG and the build scripts for the book do the exporting to PNG automatically. It's not even hard, just one command-line call.

The LiferayIPC is a Vaadin component that you add in the main window (layout). The inter-portlet communication is done between the client-side widgets.

LiferayIPC liferayipc = new LiferayIPC();
layout.addComponent(liferayipc);
...
liferayipc.sendEvent("hello", "This is Data");

To receive an event in other portlets, you define a LiferayIPCEventListener as follows:

liferayipc.addListener("hello", new LiferayIPCEventListener() {
    public void eventReceived(LiferayIPCEvent event) {
        getWindow().showNotification(
           "Received hello: " + event.getData());
    }
});

The add-on package includes this nice demo WAR that you can just deploy in Liferay. It demonstrates how to send and receive inter-portlet events.

You can combine this client-side communication method with server-side inter-portlet communication, for example, by using portlet session attributes. You may need to do that for security reasons or if you need to send large amount of data. In such case, you just send a client-side notification that says: "Hey, there's new server-side data available, read it now."

So, go ahead and read the new section in the Book of Vaadin. It explains everything. If it doesn't...go and ask in the Vaadin forum.

Theme Development in Liferay

2011-11-28T04:24:00.000-08:00

After installing a portal system (or web server), the first thing you want to do is create a simple web page and start customizing the theme. Unfortunately, Liferay's documentation is terribly unclear about that.

Well, its suggested normal method is to choose from a number of ready themes. Umm, NO! Nobody wants to use a theme like that. Every organization has their own colors and graphics and if not, you nevertheless want to have something that you have in your mind, not what some others have.

So, the "advanced" method is to develop a custom theme, which you can then deploy in Liferay. The Liferay plugin for Eclipse supports that. However, that method is way too heavy for me at least. I don't want to run a separate development Liferay on my machine, as I don't have enough memory in my home computer to do it fast enough. I want to work directly with the production server. I also suspect that deploying a theme takes longer than a second.

So, what I wanted to do was to just edit some CSS in the web server, save, reload, and see the result. This method of themeing does not seem to be documented anywhere, or at least it's hard to find. Anyhow, the easy way seems to be to edit the custom.css file in "classic" theme. The full path to the file is tomcat-x.x.x/webapps/ROOT/html/themes/classic/css/custom.css.

The problem with that was that just editing it did not reload it immediately, or actually ever. Even restarting the serve does not reload it. I found some suggestions for properties that would need to be set in the portal-ext.properties, but they didn't quite work in my case.

Finally, I found the solution right there in the Liferay's installation package. The file tomcat-6.0.29/webapps/ROOT/WEB-INF/classes/portal-developer.properties has the following settings for development:

theme.css.fast.load=false
theme.images.fast.load=false

javascript.fast.load=true
javascript.log.enabled=true

layout.template.cache.enabled=false

browser.launcher.url=

combo.check.timestamp=true

freemarker.engine.cache.storage=soft:1
freemarker.engine.modification.check.interval=0

openoffice.cache.enabled=false

velocity.engine.resource.manager.cache.enabled=false

com.liferay.portal.servlet.filters.cache.CacheFilter=false

com.liferay.portal.servlet.filters.themepreview.ThemePreviewFilter=trueroot

I don't actually know which of the settings are really necessary, but now it seems to work great. Just edit the file and it's reloaded immediately.

Installing Liferay on MySQL

2011-11-20T00:32:00.000-08:00

I'm installing a new web server for the astronomical association Turun Ursa. As we support Liferay at work, using it for this purpose may have some synergy so that I don't need to learn yet-another-portal. Well, not that Liferay is the easiest to learn. Anyhow, it also makes it possible to develop some portlets if I ever need one. I think I may need to redo my planetarium application again in Java - the old C++ version for Sparc stopped working some time ago after some upgrade at the university.

Anyhow, the instructions for doing the proper installation were rather unclear as they didn't really tell much details about creating the database. They referred to a "Chapter 6", which didn't exist at all. I guess they referred to these instructions, which were more helpful. Anyhow, below is a brief summary.

The setup is done here using the database root user. Some Liferay example used that though I don't really see why. Creating a separate Liferay user might be nicer. Well, I have Liferay running as a root anyhow to get it running in port 80, so the database user probably doesn't make it significantly safer.

Shut down the server
```
# ./tomcat-6.0.29/bin/shutdown.sh
```
...and wait for a bit.

Move the 7Cogs demo data generator away

# mv tomcat-6.0.29/webapps/sevencogs-hook/ .

Create the database with a database command and not using the mysqladmin to be able to set the character set. It's probably important.
```
# mysql -prootpassword
mysql> create database lportal character set utf8;
Query OK, 1 row affected (0.00 sec)
```
The MySQL configuration stuff is explained in this wiki.

Create a portal-ext.properties file in the Liferay installation directory with content:

#
# MySQL
#
jdbc.default.driverClassName=com.mysql.jdbc.Driver
jdbc.default.url=jdbc:mysql://localhost/lportal?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false
jdbc.default.username=root
jdbc.default.password=rootpassword

Start up the server
```
# ./tomcat-6.0.29/bin/startup.sh
```

Follow that it starts OK

# tail -F tomcat-6.0.29/logs/catalina.out

Well, rather trivial but took me a bit time as the documentation was a bit...hazy.

Other notes:

When first logging in, create a new user account with your own name. Then log out and log in as user "test@liferay.com" with password "test" (this information was not really obvious in the instructions either). Go to Control Panel -> Users and give yourself the Administrator role. You need to remove or disable the "test" account at some point...

Windows behind the background

2011-11-10T02:04:00.000-08:00

I've had a lot of trouble with my dual screen setup with Kubuntu 11.10. It works like a dream when you set up it at first. The trouble starts when you turn off either or both of the monitors and then turn them on. KDE is "intelligent" so that it tries to switch between the single and dual display set up depending on what displays are on. Unfortunately, the result is rather uncontrolled and random.

So, this morning I arrived at work and opened my laptop first...MISTAKE! I should have turned on the external monitor instead. So, it did not just not work, but also got completely stuck. So, I restarted kdm from console screen. After that, the behavior got really odd. No windows were visible. I could see them in the task panel, but they were nowhere. Then, I switched to the 3D cube view and I could see them - behind the background!

So, apparently the background is no longer a "root image" in X, but a window itself, and it has a Z index. I was able to bring the windows forth by right-clicking them in the task bar and selecting "Advanced -> Keep Above Others". But that was rather nasty thing to do. Then occasionally, the panel itself would drop below the background image and using the machine became impossible. Even restarting the machine did not help, apparently the Z index of the background was saved in system configuration. Great.

I traced the problem to .kde/share/config/plasma-desktop-appletsrc. Apparently, the background is managed as a desktop applet (aka "desktop widget") and each has a zvalue parameter defined in the configuration.

I did not trace the issue further, just removed the file. Apparently others have met the same issue, hence this post.

Cross-Site Embedding Vaadin Applications

2011-03-24T07:28:00.001-07:00

I've been recently testing the embedding of Vaadin applications.
Here's the Vaadin Sampler embedded:

Well, the application isn't made for this narrow columns, but it works, and runs on a completely different server. And isn't in an iframe.

Essentially, all you need to embed an app is:

    <script src="http://demo.vaadin.com/xsembed/getEmbedJs" type="text/javascript"></script>

However, if the app has 100% vertical size, you must put it in an element with a defined height. You may also want to set the width in some cases, such as in the example above:

<div style="height: 600px; width: 400px; overflow: auto;">
  <div style="height: 500px; width: 800px;">
    <script src="http://demo.vaadin.com/xsembed/getEmbedJs" type="text/javascript"></script>
  </div>
</div>

Vaadin Refcard updated

2010-02-10T04:00:00.000-08:00

I made a few minor corrections and additions to the Vaadin Refcard, which was released last week at DZone. If you already downloaded it, you might to get the new version.

Vaadin is available under Apache License 2.0
For Item editors, such as the Form component, you can set the item data source with setItemDataSource()
For Container editors, such as the Table or Tree components, you can set the container data source with setContainerDataSource()
The book cover on the last page was updated - the cover for Vaadin 6.2 edition is blue
The text in the last page was reordered to be more correct

The refcard could be updated next after Vaadin 6.3 is released in the late spring. The major features are use of GWT 2.0 for the client-side, drag'n'drop, and support for portlet flat mode.

Vaadin Refcard is out!

2010-02-01T10:46:00.001-08:00

The Vaadin Refcard is finally out at DZone. Download yours now or browse the card on-line.

If you're a newcomer to Vaadin or interested in finding out what it is, the Refcard offers a great introduction to Vaadin. Much of the reference material should be also useful to those already familiar with Vaadin.

The six pages of the Refcard present a well thought summary of the most essential features of Vaadin, which are covered in detail in the Book of Vaadin. Much of the reference information is presented in tightly packed diagrams; just look at them for a moment and I hope you'll see the genius of Vaadin as I do.

The contents are:

Creating an Application
Components
Layout Components
Themes
Data Binding
Creating New Components

As Vaadin is developing fast, I hope to make updates to the card occasionally. So, especially after update releases (Vaadin 6.3 is coming next), please look out for an update of the card.

More Information:

Client-Side Validation for Vaadin

2010-01-08T06:33:00.000-08:00

I recently created a new component for Vaadin, the CSValidationTextField (suggestions for a better name are welcome), which is an extension of the regular TextField. It can validate the input with a regular expression and/or a JavaScript program.

Of course, you can't trust the client-side code for validation, so you need to validate the input on the server-side as well. For regular expression validation, you can use the same expression for the server-side RegexpValidator. For example:

// The Finnish Social Security Number final CSValidatedTextField ssn = new CSValidatedTextField("SSN"); String ssn_regexp = "[0-9]{6}[+-A][0-9]{3}[0-9a-zA-Z]"; // The client-side validation ssn.setRegExp(ssn_regexp, "000000-000A"); ssn.setAllowInvalid(false); // Prevent typing invalid values // The server-side validation ssn.addValidator(new RegexpValidator(ssn_regexp, "Invalid SSN")); ssn.setRequired(true); ssn.setRequiredError("SSN is required"); form.addField("ssn", ssn);

Matching partially filled values has some problems currently. If you disallow typing invalid input, as in the example above, the regexp validation currently needs an "example" and is restricted to only fixed-length fields. I'll have to look if the JS regexp matching can somehow report if there's a partial match. If you allow invalid input, there are no restrictions and the fixed-length example is not needed.

You can validate with JavaScript just as easily:

CSValidatedTextField postcode = new CSValidatedTextField("Postal Code"); postcode.setJavaScript("if (value >= 0 && value < 10000)" + " \"partial\";" + "else if (value >= 10000 && value <= 99999)" + " null;" + "else" + " \"Postal Code must be a 5-digit number between 10000 and 99999\";"); layout.addComponent(postcode);

The server-side validation needs separate Java code though, unless you use some JavaScript execution library.

You must validate the input on the server-side as well, as the client-side validation does not actually prevent inputting invalid values (unless you set setAllowInvalid(false), but that's a bit restricted at the moment).

See the example source codes (link below) for more examples.

See:

The Demo (editors, some validation examples)
API Documentation
Source Repository
Example Souce Code

Included are also handy interactive editors for the regular expressions and JavaScript validators, as you can see in the demos.

The component is not packaged yet, so using it may require some effort. I'll have to look into the packaging.

Playing in Order

2009-10-29T11:26:00.001-07:00

I've been fighting with my E90 for a few hours to get it play the MP3s of the Lord of the Rings audio books in proper order. It's not so easy.

First, I had to set proper track and disk information in the MP3s. This was easy enough with the Amarok music player for KDE. It allows easily editing the metadata, also in a group of files. There was some hand work to edit the track numbers, which took about half an hour for the 20-disk set.

Another and a more strange problem was that my E90 orders the MP3s to play in the directory order. The directory order is a low-level technical order in which files are stored in a directory (folder), and is not necessarily the same as the alphabetical order. Apparently, in FAT, the directory order is the order in which the files are written. In Linux, however, the directory order can be about anything, depending on the file system. In the EXT3 filesystem, the order seems pretty random and is probably the order of some hash table.

Unfortunately, every copy program, such as Konqueror, scp, or the command line "cp", copies the files in the directory order, so it's not possible to trivially copy files to a phone or MP3 player in alphabetical order. One solution is to use a specially ordered copy operation from command-line.

In my case, I had the MPs of the 20 CD audio book ordered under per-disk directories, so I had to use the following somewhat complex copy command:

for dir in `ls /home/magi/data/mp3/Sormuksen\ ritarit/`; do
    echo `ls -Q /home/magi/data/mp3/Sormuksen\ ritarit/$dir/*.mp3` "/media/Magi\ 2G\ u1/Sounds/Digital/Sormuksen\ ritarit/$dir/" | xargs cp -v
done

An alternative would be to first write everything to the player in any order and then use the "mv" command to move the files to a temporary directory, and then remove the original directory and rename the temporary directory as the original.

Moreover, if you have earlier written the MP3s to E90 in one order, it remembers the order even if you rewrite the files in another order. You first have to remove them from the player application, then write them again in the proper order, and only then you can play them properly.

References
Some other solutions in Linux Forum: Change the order of files in a directory.

Vaadin Cheat Sheets

2009-10-27T16:00:00.001-07:00

I designed the Vaadin Cheat Sheet last summer. I really like it, especially its style that follows the Vaadin graphical style.

During the last few weeks, the demand for the cheat sheets has been growing. There's one on display at the ICT building in Turku, where you can get a free copy of Book of Vaadin. This week, I got some more lamination pockets to make more laminated cards. The new pockets with matt finish are nice, you can even write on them with a pencil.

I somewhat enjoy laminating the cards. Even though it's purely manual labour, it's quite relaxing to work manually by a routine for a change and not have to think or design anything. This also means that I can't make the cards by the thousands, like the books. Nevertheless, I try to make enough for IT Mill customers, some community members, and for special promotion purposes.

Just today, I corrected a small error: the CSS style class of the ComboBox component is v-filterselect, not v-combobox. There might be other similar errors.

N900 vs iPhone

2009-08-27T15:38:00.000-07:00

Hi, I collected a rough list of some features. The list is not exhaustive and there may be some important features in software and interoperability that are not covered. Please comment and I'll add new notes.

	N900	iPhone 3GS
Basic stuff
Size	111x60x18mm	116x62x12mm
Weight	181 g	133 g
Screen Size	3.5 inch	3.5 inch
Screen Resolution	800x480	320x480
Screen Colors	65536 colors?	262144 colors
Screen material	plastic?	glass
Touchscreen tech	resistive	capacitive
Touchscreen fingers	one	two
Built-in keyboard	yes	no
Screen keyboard	yes	yes
Computing
Processor	ARM Cortex A8	ARM Cortex A8
Processor speed	600 MHz	600 MHz
Graphics accelerator	PowerVR SGX	PowerVR SGX
RAM	256 MB	256 MB
Virtual memory	768 MB	none
SSD storage, internal	32 GB	16/32 GB
External storage	up to 16 GB (MicroSD)	none
Software
Operating system	Linux-based	Darwin-based
Web Browser	Mozilla-based	Safari
Zooming	yes, one-finger	yes, two-finger
Adobe Flash support	Flash 9.4	none
Java	possibly	none
Skype	yes	no
Video formats	mp4, .avi, .wmv, .3gp	mp4, m4v, mov?
Video codecs	H.264, MPEG-4, Xvid, WMV, H.263	H.264, MPEG-4?
Music formats	.wav, .mp3, .AAC, .eAAC, .wma, .m4a	m4a, ...?
Open application development	yes	no
Music/software store	Nokia Ovi	iTunes, App Store
Operator lock	no	yes
Misc hardware
Camera	5 Mpix	3 Mpix
Video recording	yes, 16:9	yes
Video size and rate	848x480 @ 25fps	640x480 @ 30fps
Microphone	stereo	mono?
Geotagging	yes	yes
GPS	A-GPS	A-GPS
Cell-based navigation	yes	yes
Accelometer	yes	yes
Wireless network	802.11 b/g	802.11 b/g
Video out	yes (PAL/NTSC, WLAN/UPnP)	yes
FM transmitter	yes	no
Headphone connector	3.5 mm	3.5 mm
Compass	no	yes
Replaceable battery	yes	no
Other
Designed in Finland	yes	no
Jesus uses it	no	yes

First Impressions with Suunto Core Wristop Computer

2008-07-27T05:04:00.000-07:00

I bought a Suunto Core "wristop computer" yesterday and as usual, I wish to write a bit about my first experiences with the device. So far, the compass is very problematic and the temperature meter is useless. The compass is the most important feature for me and it really should work.

Suunto Lumi was my second choise and just fifteen minutes after buying the Core I regretted not buying Lumi instead. Core is HUGE, basicly a UFO strapped to your wrist and distinguishable from at least lower Earth orbits. Lumi is essentially the same device but it's half the size, so it's more of the size of a conventional men's watch. For some strange reason, Lumi is marketed for women. While the Suunto Core marketing talks about traveling in wilderness and scaling mountains, Lumi marketing talks about finding directions in rock festivals. Ok, the wrist strap is a bit ornamental in a feminine way and a regular black strap might be more mannish, but Suunto sells separate straps so that should not be a problem. The only missing features in Lumi are the rotating bevel (it's useful) and the depth meter (very rarely useful). And Lumi is 50 euros more expensive than Core (250 vs 200 euros).

In this article, I will go through the main features.

Construction

As I said, it's HUGE. The only reason that I can think of for the size is that you can use it with the gloves on. Strangely, the Suunto webpages do not advertise this reason. The size is a problem in so many ways. It doesn't really fit under the sleeves and it makes more difficult to don a jacket. As I need the watch when traveling, I don't want my watch to draw the attention of the natives who make 30 euros a month and the watch worth their six months salary (not that my pro-looking camera equipment draw any attention). On the other hand, I wish I had had the watch when I went snorkling in the coral reefs at Keys. Anyhow, I would have preferred Suunto Lumi.

Despite the size the watch is really light. The size is apparently just empty space and not because the electronics require that much space. It's not light enough to float though - I tried. The wrist strap is a bit gelatine-like material and it's somewhat difficult to open the lock. You can not adjust the strap, or more accurately, you have to adjust it every time you put it on your wrist. There are so many holes that it's not easy to know which one is the best. Because of the gelatinish material, the skin under the strap sweats like a pig. The reason to have a metal wrist strap is not that it looks pretty but because it conducts heat.

Another problem with the strap is that it's positioned so that it is not possible to put the watch on a table surface (see the figure below). This makes use and especially calibration of the compass very very very difficult, because the compass must be leveled very precisely. Also, the back-side of the watch is nowhere near flat. It also makes setting compass direction on a map much more difficult. Have the designers actually used the watch?

Some people at Suunto discussion forums complained about too loose or tight compass bezels. The rotation stiffness of my bezel is excellent, but in small scale the bezel is loose and moves even if you touch is only lightly. The bezel has 60 steps so for compass purposes the steps are 6 degrees. Unfortunately, the compass scale is not written in the bezel and it only has letters indicating the cardinal directions. Some other models have a scale.

The glass is mineral glass and it is a good thing that it is not plastic as most cell phone displays are. Plastic displays scratch really easily and they are full of scratches in a year or two. The most scratch-resistant glasses are the sapphire-coated ones, but ordinary mineral glass is usually enough. The surface is convex and looks good. If you end up stranded in an island, there's no point in removing the glass and using it as a burning lens to light your campfire, as you can just use the reflection for the same purpose (if it only was reflectively coated). Oh well.

The Clock

The clock is such a basic functionality that it is hard to screw it very badly. I have not used the watch long enough to say how well it keeps time. At first, I thought that the watch lost time very fast (10 seconds per day), only to notice that it was my computer's clock that lost time. When comparing to accurate time, the Core seems to have gained about 3 seconds in 4 days, which is not good, but I'm again not certain that I set the time correctly in the first place.

The clock has a number of views: just time (hh:mm), time and date, time (hh:mm) plus seconds, time (hh:mm) plus time at a secondary timezone, time plus sunset/sunrise times, timer, countdown timer. Why are there so many views with just minor differences? What's the use of showing just time and not date or seconds? Why isn't there a view that shows all time (hh:mm:ss) and date in one view?

The only really annoying thing is the countdown timer. I need such a timer almost daily for foodmaking purposes. Setting the countdown time requires that you gough the menu: click Menu (2s) - select time-date (2 clicks) - select countdown (2 clicks). Then you can set the countdown time. Why can't you just set the countdown time in the watch view?

The Altitude Meter / Barometer

The altimeter in Suunto Core is supposed to have resolution of one meter. I haven't been able to measure this very accurately yet, but it seems believable. I used the altimeter in one measurement to measure the relative height of two pillars 400 meters apart and it gave constant readings within some 2 meters. The problems are in calibration. Core allows you to calibrate the altimeter in two ways: by a known altitude or by local atmospheric pressure at sea level, which you can get from weather stations.

The problem with the calibration by the atmospheric pressure is that Suunto Core allows setting the pressure with the resolution of one hPa, but the difference of one hPa is 27 meters in altitude! Weather stations give the pressure with one decimal. The advertised one meter altitude accuracy means that Core should be able to measure pressure with about 0,03 hPa resolution, so why can't I calibrate it even with one decimal?

Another problem with the altimeter and barometer mode is that it's a lot of work to switch between them. Switching between them through the menu (which requires 2s press to open) takes 10 button clicks in all. Not very easy! There is an "automatic mode" though, which should have the altitude meter on when the pressure changes fast (i.e. when you're moving vertically), and barometer on when you're stationary, but it takes a few (2-3) minutes to switch to the altitude mode and a lot longer (15-30 minutes) to switch back to the barometer mode.

The altiude/barometer mode also shows the temperature. Unfortunately, the body temperature affects the reading strongly, so when you're wearing the watch in your (naked) wrist, it will show 30-33°C if real temperature is 20-30°C. I did a small experiment: digital thermometer shows 28,9°C and Core shows 30°C. I put the watch in my wrist and wait 5 minutes. It will show 32°C. So, it's basicly useless when you're wearing it. Taking it off doesn't help very quickly and it takes maybe 15-30 minutes to stabilize. I would really have expected that the watch would compensate for the body temperature at least a bit, possibly by measuring the temperatures from back, middle and front sides of the watch and extrapolating the temperature. Or just by using a thermometer that measures the air temperature with infrared or something. I suppose there might not be a perfect solution, but currently it's just bad.

The Compass

The compass has been nearly useless so far, because it's highly sensitive about tilting and loses calibration all the time. So, while the specs say that it should be accurate to 1 degree, I would say that it's accurate to maybe 180 degrees, because you never know when it has lost the calibration completely or just partially. It doesn't need to be accurate to one degree and some 5° would be acceptable, but now the accuracy is much worse.

The compass is really the most important feature of the watch for me. I need it for urban orienteering when I'm traveling and for rough directions for amateur astronomical purposes. For example, I want to know the rough direction of the Sun or the Moon or some planet or a star that I see in the horizon. When I'm gazing for satellites and there's a predicted satellite flash at a certain time, I need to know the exact direction where it will show. If I see a meteor (or even a fireball), I need to be able to estimate its radiant. At night, I can usually see the directions from the stars very easily, but in daytime or in dusk I really can't. Finding Venus or Mercury from the sky during daytime requires knowing the exact direction.

Occasionally, I get happy when the reading looks reasonable, but then I realize that it's the same as a dead wall clock being correct twice a day. Typically, the error is about 20-50 degrees, so in a real situation, you would not really know that there's something badly wrong. For note, I have calibrated the declination, and it's not huge anyhow (+6.5°), so it's not that. Reading the direction at one place can give steady readings, but fifteen minutes later it gives totally different readings. Sometimes the compass goes to an oversensitive state in which, if I turn 90 degrees, the reading changes 180 degrees.

The problem may be that the compass loses its calibration very easily and needs to be recalibrated, in practice every time you use it. The watch doesn't actually allow recalibrating the compass the same way as it was calibrated on the first use. The manual says that you can recalibrate it just by rotating it slowly around for a full circle, but this method is a bit uncertain as the watch doesn't actually indicate when it has been recalibrated. The watch needs to be kept exactly leveled during calibration, but the badly shaped wrist strap (see above) makes it impossible to level the watch on a flat surface, so you need for example a bottle to put it on. But you can't carry a table and a bottle around the world in your backback, can you?

The User's Manual says that the compass must be held leveled with the ground for it to be accurate. That' s an accurate statement, even if the compass is not. Tilting the watch even slightly can change the compass reading radically, so that a 1° tilt can sometimes cause 2-3° error in the reading. Sometimes, possibly if calibrated better, the compass is less sensitive to tilting and behaves even reasonably well. The manual says that the watch would warn about tilting (Finnish manual says - English doesn't), but it doesn't give any warning for me however much I tilt or not tilt the watch. I've experimented a lot with this.

The sensitivity to tilting creates a practical problem: leveling is not easy. You can level the watch most accurately by looking at it horizontally against the horizont, but then you can't read the compass reading without lowering it. The watch isn't really designed to be held leveled. The wrist strap prevents from actually putting the watch on table. I've found a cardboard tube from an empty toilet paper roll to be useful for leveling the watch.

Pressing the light button creates a really weird light show: the light goes off and on about two times a second so it's flashes quite annoyingly. The display font in the compass mode is really ugly.

Other

A hint for Finnish buyers: Kello- ja kultaliike Suominen in Turku sells these for 200 euros while other places demand 250. Also the price for Lumi is 50 euros cheaper than elsewhere.

Disappointment with Sony Handycam Software

2008-07-01T12:57:00.000-07:00

I purchased Sony SR-12 in April 2008. I have earlier written about my first impressions and compared the camcorder with my previous Sony camcorder. So far, I have been busy just filming my travels and left watching and editing for later time. One reason for this is that I'm practically unable to watch any of the videos that I have recorded. The SR-12 comes with no less than three Windows utilities:

Handycam Utility - to copy videos to computer and DVDs
AVCHD Player - to play video from the cam or from DVD
Picture Motion Browser - to organize, edit and publish videos

All of them fail in some respect and are terribly problematic.

If you like stories of frustration, hopelessness, anger, and an occasional burst of masochistic laugh, read on.

Handycam Utility

The Handycam Utility is the software that pops up when you plug a USB cable into your camcorder. In theory, it allows you to watch videos from the camera, copy them to your computer, or to create DVDs from the videos. It fails miserably in most of these tasks. Let us go through the functionalities of the utility.

Video Playback (AVCHD Player)

You can launch the AVCHD Player from the Handycam Utility or from the desktop icon. The player allows you to watch videos from the camcorder either from the camcorder or from AVCHD DVDs. This is really what you would expect. The player has, however, a few problems. Probably the most annoying one is that it pauses for about a half a second between each shot. This is especially annoying if your shots are very short. You don't get that on TV, you don't get that on DVD, you don't get that when you play back the recordings on the small screen of the camcorder, so why do you get it on this player? Actually, the two other player softwares that come with the camcorder have the same problem, but the delay is much bigger in them.

Another problem is that once you start the player, it starts indexing the shots on the camcorder. Indexing one shot takes about 3-5 seconds. I myself take usually well over 1000 shots during a week's travel, so indexing the shots takes about 4000 seconds, that is, over an hour. Currently, I have some 3000 clips stored on the camera, so indexing would take about 4 (four) hours. That's every time you start the player, as it doesn't remember the index. The speed is a bit strange as the copy-to-pc functionality of Handycam Utility indexes the pictures maybe twenty times faster.

Third problem is that the clips are shown 6 (six) clips a page on the screen. You can click on arrow symbols on a page to change to the next page, but with my 3000 clips, it means 3000/6=500 screens! So if I want to get to a page somewhere midway, I have to click the arrow 250 times. No worry, changing page takes maybe 0,5 seconds so it "only" takes some two minutes to get there. This absurdity makes it certain that the developers of the software have never themselves used the software, not at least with their own camera. Or they only record very very long clips, half an hour and so. "Oh, we have tested the program with a large number of clips, twenty!"

There is also a problem regarding video quality. Sony SR-12 is supposed to record 30 frames/s. I dare to say that the player plays maybe half of that. The problem is clearly visible when someone walks on the screen. The playback is far smoother when you play it from the camcorder to TV, or in Linux after conversion.

You can also use the AVCHD Player to play AVCHD DVDs that you can burn with the utilities. The problem with DVD playback is that the DVD index shows the clips by date. Typically, you have just one day in the DVD, so the DVD index has just one item. The problem is that you can easily record 200 shots a day (that's my average when traveling), but the software does not have an index to allow you to jump to a certain shot easily. You just need to click the Forward button 200 times to get to the end. Changing to the next clip is not immediate, so it can take a few minutes to get to the end. It's really frustrating.

PC Backup

The PC Backup functionality allows you to copy your videos from the camcorder to your PC. That's handy. My problem with the program is that I have only 20 GB disk space on my laptop (I have a 80 GB disk but I have reserved 60 GB for Linux). Windows Vista takes about 18 GB, so I have just 2 GB free, while I need tens of GBs to store my videos. This should not be a problem, as I have a 120 GB USB disk that I can attach to my laptop. The utility actually allows me to configure the folder where the videos are stored. All good? No. After I change the folder to my external USB disk and click the Ok button...it fails and leaves the target folder unchanged. I have tried everything, but it completely refuses to make the "PC Backup" to a USB disk attached to my PC.

One Touch Disc Burn

This feature allows you to burn all the videos on your camcoder to an AVCHD format disk. Well, ok, virtually no DVD player plays AVCHD disks, but I knew that beforehand, so I can't complain. My main purpose anyhow in burning such DVDs is to get backups.

The problem is that the utility can only burn everything on the camcorder to DVDs. There's no option to start from a certain disk or a certain date or anything. You can only burn all or nothing. That would not be a problem if it was just one disk, but if you have, say, 80 GB (6 hours) of recordings on your camcorder, burning them all takes 20 DVDs. You can burn them in just one go, so if there was a problem at some point, you have to burn everything from the beginning. And if you go out and shoot more, you have to burn the earlier 20 DVDs again. Really.

The functionality is even more problematic because it is a bit slow, to say the least. Burning a DVD takes a bit over 10 minutes. So, burning 20 DVDs takes 200 minutes or 3 h 20 m. Plus time to change the disk, of course. So, please remember to have to plug in the power cord to your camera before recording, because otherwise your battery will go empty and you'll have to begin again...

Camcorder Issues with SR-12

One horribly serious problem with Sony SR-12 is that it doesn't store any unique number for the video clips. The name of a video file stored on the hard drive of the camcorder is just a sequential number starting from 1. If you remove a video from the middle, all the numbers of the later files are reduced by one. So, if you clean up files on your camcorder, you have to remove them all at once.

The camcorder actually stores the date and time when you shot the clip as the modification time of the file. So you could order the clips by date? Not possible, because if you edit a video on the camera and, say, split it in two, the split files will have modification time of the editing operation. Of course...

Linux Issues

I use Linux 97% of time and only use Windows for playing some games. Unfortunately, the Sony camcorder utilities are not available for Linux. The camera itself is just a USB memory, so by just plugging it in, I can easily copy the video clips from the camcorder to my computer. The problem is in actually watching them as no Linux video player currently supports the M2TS video format. No problem, there are ways to convert the M2TS format to formats that I can play on Linux. The problem is that so far, all there ways have failed for me.

The article Transcoding MTS/M2TS AVCHD Video Into AVI Files with Free Software provides excellent instructions for transcoding regular HD (not FullHD) M2TS AVCHD video files to H.264 AVI. I did not enjoy the shell scripts so I wrote a very similar Python script for my purposes.

So far, the ffmpeg solution either loses audio altogether or has severe audio synchronization problems. Ffmpeg doesn't have built-in scaling, but I would want to scale my videos to PAL 720x572 resolution or some other resolution lower than FullHD's 1920x1080. Regular YUV image conversion utilities would do, but using them is a bit cumbersome.

Mencoder provides another solution and works much better. Audio is ok and I'm able to scale the video as I like. The problem is that the resulting video is "jumpy": movement of people is not even but looks somehow rushed. This is probably because of "duplicate frames" that occur during the conversion. Both the input and output are 30 frames/second so there should not be problems with that, but there probably is some .

I had a really well-working video editing path with my old Sony DV camera, with dvgrab utility for grabbing the video data and Kino for editing it. I really would like to reach that same level with my new Sony.

Summary

The software that comes with Sony SR-12 camcorder is, to a large degree, unusable for the typical use for which you would expect it to work. SR-12 is not a low-end beginner model, but while definitely not a professional, it is a low-end prosumer model. As such, you expect some usability from the software. A prosumer can expectedly take thousands of video clips and wants to store, convert and edit the clips and use some video editing software to make amateus movies. However, even the basic functionalities are severely lacking.

I certainly hope that Sony comes with major updates for the Handycam Utility software. Otherwise the problems with the software make the camcorder themselves unusable.

Six Years of Evolution of Sony Camcorders

2008-03-29T11:50:00.000-07:00

I bought my previous camcorder, Sony DCR TRV50 in 2002. At the time, it was the top model of the Sony consumer camcorders, as is my new Sony HDR SR12. Let us look what has gone better and what worse.

Let us first look at the main features for which the SR12 is marketed.

Feature	TRV50	SR12
Resolution	PAL 720x576	FullHD 1920x1080
Video size	13GB/h	8GBh
Audio	2+2 channels	5.1 channels
Recording Media	MiniDV	HDD
Recording Time	1h(SP) - 1.5h(LP)	15h(FullHD) - 50h(HD/LP)
Price	2050€	1350€

SR12 clearly wins by all of its main features: FullHD (1920x1080) image resolution, 5.1 surround audio, and 120GB hard disk storage capacity. The HDD media also wins for the easiness to browse the media. With MiniDV tapes, you have to fast-forward and rewind the tape to get where you want, and in SR12 the camera simply displays an index of all shots. You can edit the individual shots and delete them freely, which is impossible with MiniDV, at least without scissors and glue.
The real questions are, are the actual image and audio quality actually any better, and what about the other features?

Physical features

Feature	TRV50	SR12
Screen size	3,5" 3:4 LCD	3,5" 9:16 LCD
Viewfinder	Yes, tilting+telescoping, bigger	Yes, tilting
Neck strap	Yes, 2-point	Not included, 1-point
Lens cover	plastic cap	automatic shutter

The screen size is same for both cameras, about 3.5" with equal width, but TRV50 has a 3:4 screen, which makes it higher. The display in TRV50 was more matt and SR12 has a more polished screen surface. Polishing is generally bad, because it reflects light, but modern coatings help with the problem.

The viewfinder in TRV50 was much bigger than in SR12 and it was possible to telescope it further back. This is important if the battery is bigger than normal, as is my case. A good viewfinder is very important in bright daylight.
As you may see, the battery is also somewhat smaller in SR12. Well, the SR12 battery is the standard battery, while in TRV50 I have the biggest battery. The battery life with the SR12 standard battery is about 1,5 hours, while the TRV50 battery has 6,5 hours. The 1,5 hours is ridiculously short, as I have experienced that the 6,5 hours is barely enough for a day of touristing. You definitely need a bigger battery, the biggest one is some 150€. Unfortunately, it will also make the camera bigger and, as it has no telescoping viewfinder as TRV50 has (see the picture), using the viewfinder is more difficult, at least when it is leveled and not tilted as in the picture above.

Performance

The FullHD SR12 has five times as many pixels as the PAL TRV50. But does this mean better actual resolution? The lenses are about the same size, with the lens of SR12 possibly a bit smaller, so the optical resolution may not be any better. On the other hand, in digital cameras, the resolution has grown just fine while keeping the optics about the same size. So the size might not matter.

Comparing the actual performance of the cameras would require careful testing. I may do such tests later, but for now I will just present a rough evaluation.

The actual resolution of video in SR12 is clearly better than in TRV50, but also obviously not the five times as good as the number of pixels would suggest. I estimate that the actual resolution is only about two times better.

One bigger problem is that as you five-fold the number of pixels, you drop the level of light for each pixels to one fifth. It is not probable the the light detection efficiency of CMOS chips has five-folded in six years. However, there is some advance in noise reduction in image processing. My observations support this logic, the SR12 seems to perform much worse in low-light conditions.

SR12 is has an optical Super SteadyShot image stabilizer, while TRV50 has non-optical. While optical stabilizers are generally considered much better than non-optical, I can not see real difference between the cameras. One reason may just be the size: the smaller and lighter SR12 has less momentum to prevent shaking in the hand, and the optical stabilization could merely compensate for the size. I have noticed some difference though. TRV50 apparently stabilizes small vibrations well, and SR12 stabilizer also bigger. But the stabilization in SR12 makes the video very jerky when the camera moves more than the area of stabilization, resulting in sudden jumps in the video.

Extra Features

Feature	TRV50	SR12
Bluetooth	Yes	No
Internet	Yes: Web, Email	No
Face detection	No	Yes
Face index	No	Bad

Yes, TRV50 actually has a web browser and email, and the Bluetooth connection for connecting to Internet via a mobile phone. I've used the web browser a countless times before I had other mobile browsers, and it actually works. Well, it is slow and the touchscreen keypad is rather bad, but it works.

SR12 has an interesting face detection feature. Yes, it actually detects faces on the video. It uses the faces for two purposes: to set focus and exposure parameters, and to make a video index out of the faces. Unfortunately, the video index is only for the particular shot: it doesn't make an overall index of faces in all shots. The face indexing is also very picky and doesn't index most of the faces, therefore making it basicly useless.

Conclusions

There has clearly been progress in six years. While the nominal resolution has grown five-fold, the increase in actual resolution may not be much better. Low-light performance may even have gone much worse. Physical features such as the display, viewfinder, and neck strap have also gone worse, though there are also advances. The biggest progress is actually in software and having a hard disk drive instead of tapes helps a lot in organizing and playing back videos.

That's it for now. I may update this article later and write other reviews about some more specific features of SR12. Especially, I will need to research the video resolution, low-light capability and the image stabilization.

See also:

Disappointment With Sony Handycam Software

First Look at Sony HDR-SR12 Camcorder

2008-03-25T16:20:00.000-07:00

Finally, I got it, my new FullHD digital video recorder, Sony HDR-SR12. This is the most high-end of this spring's consumer series from Sony.

The camera is really small, much smaller than my previous digital camcorder, Sony DCR TRV-50E, which was also very small at the time when I got it. SR12 fits well in the pocket of my winter jacket and really well in my backpack.

The package did not include a real camera bag, just a think cloth pouch, which I think is probably better than a bag. I usually keep my kamera in my backbag, where it's already protected so there's no need for thick bag, but a thin cloth is good to keep scratches out.

The optical Super SteadyShot is better than the non-optical one in TRV50, but not much better. A tripod is still an essential aid for video camera.

I was quite disappointed that the camcorder did not include any kind of shoulder strap. This is really unusual. It can be purchased as an accessory, but the fastening does not look very good. I will try to use the shoulder strap from my TRV50 or some other camera.

The 3,5" display is really crisp and clear. No wonder, as the resolution is 1920x480. The horizontal resolution probably contains separate R, G, and B pixels, so it's not really that high resolution, but very good anyhow. It's a touchscreen, so I'm worried how it will look in a year, but at least so far it has seemed to repel the dirt from my fingers quite well.

Transferring the video files to my Linux computer was easy, just plug the cable in the computer and the camera appears on desktop as an external media device. The problem was that the AVCHD video files are mts files, which are not easily opened. Luckily, I found an excellent article on Transcoding MTS/M2TS AVCHD Video Into AVI Files. Installing the software was not straight-forward, as it required some badly documented manual work. For example, the ldecod had to be downloaded from an obscurely documented location. After that, the program worked on standard definition video files with 1440x1080 resolution. I had to fix the program to work with FullHD 1920x1080 resolution. After that, I got video. I was very pleased at the quality, which really was clear image at the full resolution. Even a still image capture from the video was really crisp, even in indoor light conditions.

(Click on the image for the full-size image.)

Notice that the image above is not a still photograph taken with the camera, but a still capture from the video stream.

Unfortunately the video is 1920x1080i, that is, interlaced, just like in old PAL camcorders such as my old TRV-50. This is the most prominent visual problem that makes the video look cheap if there is basicly any movement at all in the video. Luckily, interlacing can usually be removed quite well without halving the video resolution. Apparently, the conversion software doesn't remove the interlacing, so I'll have to study the problem closer.

Playback is another big issue. No program I have can decode the AVI fast enough. I removed scaling and interlacing from Kaffeine and now it almost works if there is very little movement in the video. If I can't get the playback fixed, I have to reduce the resolution of the videos I save.

The audio is rather badly out of sync. This seems to be a common problem for people using the program, so I will have to inspect it further. It may also be caused by the speed problem, as, for example, kaffeine plays audio just fine at normal speed but the video is slow.

I also got a bluetooth microphone for the camera. The problem is that the bluetooth transceiver is ridiculously big. Well, more about that later.

See also:

Disappointment With Sony Handycam Software

More experiments with gigabit

2008-02-14T02:53:00.000-08:00

My previous experiences regarding my new gigabit network were disappointing. I studied the problem further by using iperf to generate raw traffic between my two Linux PCs and using KSysGuard and iftop to monitor the traffic. Now I got a whopping 950Mb/s, near the theoretical maximum. Well, it should have gone over 1000Mb/s, so there's still something lacking a bit.

The gigabit LAN with 1000baseT/FD link mode is supposed to be full duplex, so I tried two-way traffic with iperf. The result was, however, roughly a gigabit in total, so in effect it was half-duplex. This may be due to the PCI bus that the integrated network card uses. Normal PCI is a 32-bit bus and operates at 33Mhz frequency, giving (32*33M) roughly one gigabit of capacity. But, that is the entire bus capacity, it is shared with other devices, and full duplex gigabit would need twice that. So, ok, I got near the maximum.

The performance of Buffalo LinkStation still feels strange. With KDE's file manager as well as with smbclient, I got some 10MB/s. By actually mounting it as a filesystem with smbmount, I only got a fraction of that, perhaps some 1.5MB/s. That's 1.5% of the "gigabit" capacity! With Windows XP under Vmware, I got about the same as with Linux smbclient, some 10MB/s. I'm yet to test it with non-vmware XP more accurately, but it may be a bit faster.

Assembly of Ungoliant

2008-02-12T13:51:00.000-08:00

I updated my hardware recently, so I'll tell all the interesting details here.

I first bought a new case with 500W power supply for a motherboard I planned to buy from my fellow worker. Unfortunately, the chipset fan on the motherboard had gone bad and created great amount of problems when playing games, so I made a quick purchase at local hardware store.

Intel Core 2 Quad q6600 G0 2,4GHz processor
Abit IP35 ATX motherboard with Sata2, Gigabit LAN and so on
MSI GeForce 8800 GT 512MB OverClocked graphics card
4GB of 800MHz DDR2 RAM
DVD-RW drive (I needed a new one in black - oh the vanity)
Antec Sonata III case with 500W power supply

The Quad is the model with G0 stepping, making it ideal for overclocking. In even simple tests, some have gotten some 35% overclocking. I might overclock it to 3GHz (+25%) at some point. The MSI 8800 GT is factory-overclocked by 10%.

The Antec Sonata case was a really pleasant surprise. First of all, it looks good, which its shining black finish -- the primary reason for naming the machine Ungoliant. Unfortunately, the black finish is very tender and I got it badly scratched very easily. Maybe I should have bought some lacquer to protect it from the environment. Another pleasant thing was the organization of hard drives so that they are loaded on sleds and are easily accessible from the side. The side panel opens easily with a handle. On the back there is an adjustable case fan. On the front panel there are two beautiful blue leds, one for power and one for hard drive. This case is a dream.

So, I got the stuff, piled it on my desk at work, and started unboxing.

The Core 2 Quad processor was really small and had an enormous fan. Here, I have removed the bottom cover and all the connectors are visible. In the center, there are some small components.

The cooler element had interesting pattern. At the bottom, it had the paste already spread, so I just needed to attach it on the processor.

Mounting the processor in the socket was easy. Mounting the cooler+fan on the processor wasn't. It had to be fastened to the motherboard with four hatches and I have to say, it was really hard and needed a lot of muscle. Fortunately, my fellow worker with more experience in assembling machines helped me with the cooler a bit so I got it fastened safely.

The rest was easy. Below is the motherboard with just the graphics card missing. The chipset is cooled with a heat pipe. I later noticed that the 2x2GB RAM combs were actually just 1GB. I returned them to the store and got my 2GBs.

Below is Ungoliant ready to be transported to its lair:

Gigantic disappointment in gigabit

2008-02-11T08:02:00.000-08:00

I bought a cheap 35€ gigabit switch on saturday and some more cat-6 cables, in an attempt to get my home network upgraded to gigabit speeds. It failed.

I found out that my new "gigabit" 500GB Buffalo LinkStation was not able to provide more than one tenth of the gigabit, about 10MB/s with Samba from Linux. With FTP, I was able to get enormous 15MB/s. With Windows XP, the access may have been a bit faster, though I did not measure that more accurately, because Windows doesn't report transfer speed when copying files. Between two Linux PCs, I got a maximum of 35MB/s with scp. Curiously, with KDE file transfer over sftp connections, I got only about 10MB/s.

It appears that I was a bit too blind expecting that gigabit upgrade would be as painless as 10Mbps and 100Mbps upgrades were years back. Apparently, the current PC hardware simply can't deliver gigabit. The integrated "gigabit" network cards apparently use PCI bus, which has theoretical maximum of one gigabit (32 bits * 33 MHz). But that's only in one direction, while gigabit ethernet is usually Full Duplex, so it would require two gigabits of bus.

Blog created

2008-02-09T10:28:00.000-08:00

Ok, here it goes. I've been planning to put up a blog or ages, but haven't gotten myself to do it. One reason has been the lack of a proper software. My intention has been to put the log in my home pages, but it would require a lot of technical solutions for which I don't have the time for now.

It will be interesting to see how Blogger fits to my needs. I write in two languages and it could be that this is not suitable for that. I also write about many different topics, and would like to separate these topics from each other, while keeping some connections.

So, here it is and I'll think about the technical solutions again later in the future.