Entries Tagged as 'Linux'

Virtual machines need regular defragging, researcher says

This comes from an article on ComputerWorld, all I can say is duh!

Virtual disks require the same fragmentation as the same operating system would running on physical machines; plus if you choose dynamically expanding containers for the disk on the host, you’ll likely need to power down the machine and periodically defragment the host as well.

You’d think that an article that starts with a title like that couldn’t possible get any more asinine; well, you’d be wrong:

Windows, as well as third-party software firms, offer defragmenters to reassemble fragmented files. Fragmentation is not as large of a problem on Unix systems, due to the way that the OS writes files to disk.

Apparently the author seems to think that just because Windows includes software to defragment the file system, it must be much more susceptible to fragmentation.  He’d be right if we were talking about Windows 98 or if people choose not to run NTFS… but he and the article he references are dead wrong.

NTFS has almost identical abilities as EXT2, EXT3, and EXT4 file systems to avoid fragmentation — the difference is that NTFS supports defragmentation of the file system (and Windows ships with a rudimentary defragmenter).  In fact, if *nix file system were so impervious to fragmentation, why would the ability to defragment be one of the major feature additions in EXT4 (though not fully implemented yet)?

There are many thing about *nix type operating systems that can clearly be pointed to as superior than Windows, the resistance to fragmentation simply isn’t one; WAKE UP and live in the current millennium, we don’t need to confuse FAT16/FAT32 with Windows.

Virtual machines need regular defragging, researcher says
By Joab Jackson on ComputerWorld

Originally posted 2010-10-12 02:00:44.

PHP as a general scripting language

Invariable working smart involves automation, and automation will almost always involve scripting repetitive tasks.

Under *nix operating systems you could write a shell script, though the syntax is arcane and maintainability isn’t really achievable for a script of any complexity.

Under Windows you can write a bat/cmd script; but the language isn’t very powerful, nor is it robust.

A good choice for all operating systems is PHP5.

Yes, you will likely have to install it — but given that PHP5 is a useful tool, free, and runs on every modern computing platform that shouldn’t be a hurdle.

PHP is a rich scripting environment.  And if you’re interested in doing any web programming you can leverage your general scripting experiences to help you learn the language and become proficient.

One language to rule them all…

Well, that might be a little over the top.  PHP is certainly powerful and flexible — but still languages like C# and Java have their uses (as well an un-managed languages like C/C++).

The best way to get used to PHP is just start using it any time you’d normally write a script in some other language.  The syntax is fairly straight forward, and the available libraries/objects are rich.

For the foreseeable future I’ll be writing my scripts in PHP, and leaving shell programing, batch files, and voodoo to others.

http://php.net/

Originally posted 2010-03-27 02:00:04.

Linux – Desktop Search

A while ago I published a post on Desktop Search on Linux (specifically Ubuntu).  I was far from happy with my conclusions and I felt I needed to re-evaluate all the options to see which would really perform the most accurate search against my information.

Primarily my information consists of Microsoft Office documents, Open Office documents, pictures (JPEG, as well as Canon RAW and Nikon RAW), web pages, archives, and email (stored as RFC822/RFC2822 compliant files with an eml extension).

My test metrics would be to take a handful of search terms which I new existed in various types of documents, and check the results (I actually used Microsoft Windows Search 4.0 to prepare a complete list of documents that matched the query — since I knew it worked as expected).

The search engines I tested were:

I was able to install, configure, and launch each of the applications.  Actually none of them were really that difficult to install and configure; but all of them required searching through documentation and third party sites — I’d say poor documentation is just something you have to get used to.

Beagle, Google, Tracker, Pinot, and Recoll all failed to find all the documents of interest… none of them properly indexed the email files — most of the failed to handle plain text files; that didn’t leave a very high bar to pick a winner.

Queries on Strigi actually provided every hit that the same query provided on Windows Search… though I have to say Windows Search was easier to setup and use.

I tried the Neopomuk (KDE) interface for Strigi — though it just didn’t seem to work as well as strigiclient did… and certainly strigiclient was pretty much at the top of the list for butt-ugly, user-hostile, un-intuitive applications I’d ever seen.

After all of the time I’ve spent on desktop search for Linux I’ve decided all of the search solutions are jokes.  None of them are well thought out, none of them are well executed, and most of them out right don’t work.

Like most Linux projects, more energy needs to be focused on working out a framework for search than everyone going off half-cocked and creating a new search paradigm.

The right model is…

A single multi-threaded indexer running in the background indexing files according to a system wide policy aggregated with user policies (settable by each user on directories they own) along with the access privileges.

A search API that takes the user/group and query to provide results for items that the user has (read) access to.

The indexer should be designed to use plug-in modules to handle particular file types (mapped both by file extension, and by file content).

The index should also be designed to use plug-in modules for walking a file system and receiving file system change events (that allows the framework to adapt as the Linux kernel changes — and would support remote indexing as well).

Additionally, the index/search should be designed with distributed queries in mind (often you want to search many servers, desktops, and web locations simultaneously).

Then it becomes a simple matter for developers to write new/better indexer plug-ins; and better search interfaces.

I’ve pointed out in a number of recent posts that you can effective use Linux as a server platform in your business; however, it seems that if search is a requirement you might want to consider ponying up the money for Microsoft Windows Server 2008 and enjoy seamless search (that works) between your Windows Vista / Windows 7 Desktops and Windows Server.

REFERENCES:

Ubuntu – Desktop Search

Originally posted 2010-07-16 02:00:19.

Linux Server

I’ve been experimenting with a Linux server solution for the past couple months — I was prompted to look at this when my system disk failed in a Windows Server 2008 machine.

First, I’m amazed that after all these years Microsoft doesn’t have a standard module for monitoring the health of a system — at the SMART from disk drives.

I do have an Acronis image of the server from when I first installed it, but it would be a pain to reconfigure everything on that image to be as it was — and I guess I just haven’t been that happy with Windows Server 2008.

I personally find Windows Server 2008 needlessly complicated.

I’m not even going to start ranting on Hyper-V (I’ve done that enough, comparing it head-to-head with other technology… all I will say is it’s a good thing their big competitor is Vmware, or else Microsoft would really have to worry about having such a pathetic virtualization offering).

With a Linux distribution it’s a very simple thing to install a basic server. I actually tried Ubuntu, Centos, and Fedora. I also looked at the Xen distribution as well, but that wasn’t really of interest for a general purpose server.

Personally I found Centos (think Red Hat) to be a little too conservative on their releases/features; I found Fedora to be a little too bleeding edge on their releases/features (plus there’s no long term support commitment); so I was really just left with Ubuntu.

I didn’t really see any reason to look exhaustively at every Debian based distribution — Ubuntu was, in my mind, the best choice of that family; and I didn’t want to look at any distribution that wasn’t available at no cost, nor any distribution that didn’t have a good, stable track record.

With Ubuntu 10.04 LTS (10.04 is a Long Term Support release – which makes it a very good choice to build a server on) you could choose the Desktop or the Server edition — the main difference with the Server verses the Desktop is that the server does not install the XServer and graphical desktop components (you can add them).

The machine I was installing on had plenty of memory and processor to support a GUI, and I saw no reason not to install the Desktop version (I did try out the server version on a couple installs — and perhaps if you have an older machine or a machine with very limited memory or a machine that will be taxed to it’s limits or a machine that you want the absolute smallest attack surface you’d want desktop — though almost all those requirements would probably make me shift to Centos rather than Ubuntu).

My requirements were fairly simple — I wanted to replace the failed Windows 2008 Server with a machine that could perform my DNS, DHCP, web server, file store (home directories — served via CIFS/Samba), and active P2P downloads.

Additionally, the server would have to have fault-tolerate file systems (as did the Windows server).

Originally my testing focused on just making sure all the basic components worked, and worked reasonably well.

Then I moved on to getting all the tools I had written working (I converted all the C# code to PHP).

My final phase involved evaluating fault tolerant options. Initially I’d just used the LSI 150-4 RAID controller I had in the Windows Server 2008 (Linux supported it with no real issues — except that Linux was not able to monitor the health of the drives or the array).

I didn’t really see much need to use RAID5 as I had done with Windows Server 2008; so I concentrated on just doing RAID1 (mirroring) — I tried basic mirrors just using md, as well as using lvm (over md).

My feelings were that lvm added an unnecessary level of complexity on a standalone server (that isn’t to say that lvm doesn’t have feature that some individuals might want or need). So my tests focused primarily on just simple mirrors using md.

I tested performance of my LSI 150-4 RAID5 SATA1 PCI controller (with four SATA2 drives) against RAID1 SATA2 using Intel ICH9 and SiI3132 controllers (with pairs of SATA1 or SATA2 drives). I’d expected that the LSI 150-4 would outperform the md mirror with SATA1 drives on both read and write, but that with SATA2 drives I’d see better reads on the md mirror.

I was wrong.

The md mirrors actually performed better across the board (though negligibly better with SATA1 drives attached) — and the amazing thing was that CPU utilization was extremely low.

Now, let me underscore here that the LSI 150-4 controller is a PCI-X (64-bit) controller that I’m running as PCI (32-bit); and the LSI 150-4 represents technology that’s about six years old… and the LSI 150-4 controller is limited to SATA1 with no command set enhancements.

So this comparison wouldn’t hold true if I were testing md mirrors against a modern hardware RAID controller — plus the other RAID controllers I have are SAS/SATA2 PCIe and have eight and sixteen channels (more spindles means more performance).

Also, I haven’t tested md RAID5 performance at all.

My findings at present are that you can build a fairly high performance Linux based server for a small investment. You don’t need really high end hardware, you don’t need to invest in hardware RAID controllers, and you don’t need to buy software licenses — you can effectively run a small business or home office environment with confidence.

Originally posted 2010-06-24 02:00:09.

Formatting an SD Card

So you like many PC and Mac users (not really an issue if you use *nix and understand what you’re doing) have had a horrible experience trying to format a SD, SD-HC, or SD-XC full size, mini, or micro card for use in your device… fortunately the SD Association, the people who set standards for these cards are offering a free download of a SD card formatted program for Windows and OS-X… just use the link at the bottom of this post and go to their “downloads” section.

And a few things to keep in mind when you go out to purchase a new SD card — look at the speed ratings, the higher the “class” number on the card, the faster it is.  And take a look at the warranty, life-time warranties are always something you’ll use (flash devices have a limited life), but certainly you’ll want to get a reasonable warranty length (just in case you got the lemon off the shelf I missed).

sdcard.org

 

NOTE: There’s also a link on the SD Association to a driver to a driver Microsoft provides which may resolve issues with SD-XC memory cards when using an SD-XC compatible reader/writer.

Originally posted 2011-11-22 02:00:38.

FileZilla – The free FTP solution

If you have a need to transfer files via FTP, SFTP, SCP, etc and you prefer to user a graphical user interface on a Windows, Mac, or Linux machine — then the Open Source FileZilla is a very good solution to consider.

Just download the client, install it, and within a few moments you’ll have a connection to a server (that you can save the information for quick reuse if you like).  The interface is clean and easy to understand, and supports drag-and-drop as well as transfers from the multi-pane manager.

And, you can’t beat the price – FREE.

http://filezilla-project.org/

Originally posted 2011-09-13 02:00:47.

VirtualBox LinuxDesktop RealPerformance

The other day I installed VirtualBox OSE on my Ubuntu machine so that I could migrate over a Windows Server 2003 machine.  I wasn’t really expecting great performance since I was putting the virtual disks on a single spindle…

Sometimes you get a good surprise.

When I started up the virtual instance, it seemed very fast — so I shut it down and started it again.  Then I performed a few quick tests and I realized that not only was VirtualBox on a Ubuntu 10.04LTS Linux machine substantially faster than on a Windows 7 machine (with a faster hard disk and faster processor), but it was faster than on a Windows Server 2008 machine running Hyper-V.

The really incredible thing was that Hyper-V was running on a disk array with fifteen spindles verses a single spindle for VirtualBox.

I really didn’t have any way to do a set of rigorous tests, but what I found was that as long as the disk wasn’t saturated, VirtualBox was able to handily outperform Hyper-V on every test (read or write) that I performed… it was only when I started to push near to the limits of the drive that VirtualBox and Hyper-V had similar disk IO performance.

I didn’t evaluate how VirtualBox performed on Linux with a disk array, but my guess is that it’s simply much more efficient at scheduling disk IO than Hyper-V; and likely Linux is more efficient at disk IO than Windows period.

I’m a huge fan of VirtualBox; and if I knew now what I knew about Hyper-V eighteen months ago I would have avoided it like the plague and simply used VirtualBox or Xen as a virtualization solution.

I’ll put a more thorough investigation of disk IO and VirtualBox verses Hyper-V performance on my “TO-DO” list; but I don’t expect it’ll float to the top until this Winter at the earliest; until then my advice is choose VirtualBox (or Xen).

Originally posted 2010-08-24 02:00:27.

LibreOffice announced for Web, iOS, Android

The Libre Office project announced today that they will officially support web browsers, iOS, and Android according to The Document Foundation.

Libre Office which formed from the community open source of Open Office and the work done by go-oo.org has primarily focused on being an office suite for the current times rather than trying to compete with or take market from Microsoft Office, and expanding into venues to support phones and tablets is a reasonable path, and one that many have been asking for for quite some time.

No specific time line has been announced, but there had already been some work done on porting to mobile platforms.

Originally posted 2011-10-14 02:00:01.

Libre Office on Ubuntu

If you want Libre Office on Ubuntu and you just can’t wait until 28-April-2011 to upgrade to Ubuntu 11.04 (which should contain Libre Office), then here’s the quick way to make it happen…

 

First, remove Open Office

sudo apt-get remove openoffice*.*

Then setup the PPA

sudo add-apt-repository ppa:libreoffice/ppa
sudo apt-get update

Then do one of the following (based on your desktop manager)

sudo apt-get install libreoffice-gnome

sudo apt-get install libreoffice-kde

sudo apt-get install libreoffice

My recommendation is that you just wait and update your Ubuntu to 11.04 on Thursday — then remove Open Office and install Libre Office… but you are the master of your own computer.

Originally posted 2011-04-26 02:00:51.

USB Hard Drive Adapters

 Everyone’s making them and they come in really handy…

 Basically they’re devices you can use to access a bare hard drive.  Most of them supports PATA and SATA 2.5″ and 3.5″ drives (though some vendors require a bunch of adapters to do it).  The APRICORN DriveWire unit is clean and simple and priced around $30 (use a price search engine) or less.

I was so happy to find these units that I purchased two of them and gave away my previous ones made by another vendor.

If you’re going to routinely swap drives on and off a computer, and don’t want to spring for an external case you might be better off with a hard drive dock also available for about $30, but they don’t support PATA (PATA is not hot swapable).

If you’re going to use these units to upgrade a computer’s hard drive, remember Acronis TrueImage is a great tool (you can find shareware and OpenSource tools as well — but TrueImage is well worth the price and has many additional features that you’ll likely find useful).


APRICORN: DriveWire – Universal Hard Drive Adapter

Originally posted 2008-12-29 12:00:32.