Technology

Using Python and K-Means To Find The Dominant Colors In Images

Below you will find a re-post of a popular article I wrote for my blog in October of 2012. This article was also included in the February 2013 issue of Hacker Monthly.

While I do not have much to add to the original content of the post (beyond a few small edits for clarity and conciseness), I would just say that I had a lot of fun writing this post and hearing how different readers had found novel use-cases for the algorithm described. One redditor used the algorithm to autogenerate his console color settings based on the wallpaper he used. A hacker news reader described using a similar algorithm to generate gradients for various images and icons in his mobile application. It was also suggested that Windows may use a similar algorithm to generate the highlight around icons that appear in the taskbar. Check out the hacker news comments to read about more applications.

Since writing I have learned that scipy has a fast implementation of the k-means algorithm. In the hacker spirit I have included my own code, but for “real” applications the scipy library will have much better performance.

I hope you have as much fun reading as I did writing – happy hacking!

I’m working on a little photography website for my Dad and thought it would be neat to extract color information from photographs. I wanted to automatically categorize photos based on their color information. I tried a couple of different approaches before finding one that works pretty well. This approach uses k-means clustering to cluster the pixels in groups based on their color. The center of those resulting clusters are then the “dominant” colors. k-means is a great fit for this problem because it is (usually) fast. It has the caveat of requiring you to specify up-front how many clusters you want — I found that it works well when I specified around 3. Read more

Dear Developer: “I Want To Get Into Game Development”

Dear Developer,

“I want to get into Game Development. Currently I am a Web Developer. What should I do? Where should I go?”

Scott: I love this question, and it has many answers. But before I give specific direction, you need to do a little self reflection. First ask yourself, “What kinds of games do you want to make?” Because just like web development, with Back End, Front End, and Interactive types of environments, game development has multiple facets. Not really sure what kinds of game you want to make? That is just fine, read along and see if something calls out to you. Lets see what is behind door number one!

Read more

NerdCast #5: Follow The Drupal Brick Road

Episode Summary:

In this NerdCast we talk with two real Drupal developers in their natural habitat. We learn more about the Drupal framework and its uses, applications, and the special sauce behind its community driven updates. We discuss Drupal 8 and the future of the framework.

We close the episode with an exciting lightning round of projects Chris and Frost would like to build in a hypothetical world using Drupal.

Host: Ryan Carlson
Guest: Chris Weber & Frost Simula

Host: Ryan Carlson
Guests: Chris Black

Listen Now:
Running Time: 0:35:23 / Subscribe on iTunes

Podcast: Play in new window | Download

Tech Talk: Learning New Tricks, My Mac Experience (Developer Edition)

Developers at The Nerdery have a lot of choices to make when they start their new job. Although having a dozen different types of cereal in the cafeteria to choose from seems like a big decision to make, it rarely impacts their work performance. Each employee has a decision to make over their choice of workstation and development tools.

Scott Carpenter shares his experience when he joined the ranks of code-ninjas here at The Nerdery and how he made a very adventurous choice to “try something new”.

Learning New Tricks, My Mac Experience
by Scott Carpenter

An email from The Nerdery after I accepted the job:

“Let us know what you want to use, Mac or PC, and what software you’ll need so we can have everything ready when you start.”

Not entirely unexpected, and this is as it should be, but, what a refreshing statement! It told me, you’ll have some freedom of choice here. (Unlike the scenario where you’re not even allowed to set your browser’s home page, which has happened to me and was less than empowering.)

Well, what did I want to use? I’ve used Windows since slightly before Win 95 launched, and at home, exlusively GNU/Linux since 2008. I’m comfortable and productive with both, and was aware that Ubuntu might even have been a valid choice, but I thought: maybe it’s time to see what this Mac business is all about. Macs have good hardware and are pretty. And they have Unix under the hood, right? Okay then.

I had some doubts on my first day when I realized, I don’t know how to use this computer. This is a building full of smart people, and I’m new, and I don’t even know how to use my machine. But a computer is a computer. You know what you should be able to do; you just have to find out how to do it on this weird fruity machine. Luckily there is this thing called The Google, and there were my fellow Mac-using newbs to help me with things like finding the power button.

So what happens when you start using a Mac? I’m going to focus on a couple of keyboard-related problems you’ll run into immediately, especially if you’re a developer that uses the keyboard a lot, and you spend a lot of time in various editing windows. I’ll give you a couple of sanity-saving and life-improving tips.

But first we should talk about balance. In particular, when configuring a machine, whether hardware or software. I’ve learned there’s something to be said for accepting the defaults when possible. It simplifies your life and saves time by reducing the endless customization you’ll tend to do.

But then again, tinkering is fun. And we’re all different. It’s not just about how Steve wanted you to use a computer. It’s also about your needs, and your feelings. Instead of saying, “You shouldn’t want to do that,” or asking, “If you want it to work like Windows, why don’t you use windows?”, let’s fix the glitch.

The first thing is fairly simple, having to do with this strange “command” key. There is a control key on the outside left and right positions on a Mac keyboard, as is proper, but it’s not like the control key you know. Instead, the peculiar “command” button has usurped most of its power.

This turns out to be okay. Many of the key combos you’re used to work just fine with command. But. It’s in the wrong place. So the first thing you’ll want to do is swap the command and control buttons in “System Preferences… > Keyboard > Modifier Keys…”

It may be helpful at first to re-label the keys to remind you of the change:

This developer’s properly configured and labeled Mac keyboard.

(We’ll pause for a moment to allow for weeping from native Mac users.)

The next thing we’ll fix is the atrocious behavior of the “home” and “end” keys.

By default, the home key will scroll to the top of your document, but leave the cursor wherever it’s at. End will likewise scroll to the end. This is completely unacceptable. Here is what our fingers have been trained to use:

I found a nice solution, which I’ve extended after a bit more research. I like this method because it doesn’t require a third-party program, and it can be set per user.

You’ll need to edit the default key bindings file:

~/Library/KeyBindings/DefaultKeyBinding.dict

On my machine, the KeyBindings dir and the DefaultKeyBinding.dict didn’t exist, so I just created them. Add the entries below:

{
    /* Remap Home / End to Appropriate Behavior */
    /* Home */
    "\UF729"   = "moveToBeginningOfLine:";
    /* End  */
    "\UF72B"   = "moveToEndOfLine:";
    /* Shift + Home */
    "$\UF729"  = "moveToBeginningOfLineAndModifySelection:";
    /* Shift + End  */
    "$\UF72B"  = "moveToEndOfLineAndModifySelection:";
    /* Cmd + Home */
    "@\UF729"  = "moveToBeginningOfDocument:";
    /* Cmd + End  */
    "@\UF72B"  = "moveToEndOfDocument:";
    /* Shift + Cmd + Home */
    "@$\UF729" = "moveToBeginningOfDocumentAndModifySelection:";
    /* Shift + Cmd + End  */
    "@$\UF72B" = "moveToEndOfDocumentAndModifySelection:";
}

(Note that we’re using the command key in place of control, since we moved it to the correct position on the outside.)

You’ll need to restart an open application to use the new bindings in that app, and it doesn’t work in all applications. For example, it doesn’t work in the built-in Terminal program, but in there, you can go to “Preferences > Settings > Keyboard” and set home = \\033OH and end = \\033OF. (Make action = “send string to shell:”)

Side note: In Terminal, the command key doesn’t assume the control key’s duties. Your fingers will often forget this, pressing Command (AKA pseudo-Control) + C or D, and you’ll wonder why your program doesn’t stop.

Bonus tip: Your fingers also may want to use Ctrl + arrow keys to move left and right one word at a time, and also to use “shift” to select the words. For that:

    /* Cmd + Left */
    "@\UF702"  = "moveWordBackward:";
    /* Cmd + Right */
    "@\UF703"  = "moveWordForward:";
    /* Shift + Cmd + Left */
    "@$\UF702" = "moveWordBackwardAndModifySelection:";
    /* Shift + Cmd + Right */
    "@$\UF703" = "moveWordForwardAndModifySelection:";

(Again with Cmd because we’ve moved it into the Ctrl spot.)

More useful info here.

I also spent time learning new habits and adjusting to “what is,” but these changes to “what should be” helped me feel much more comfortable in the new environment, along with getting a proper ergonomic keyboard and trackball:

Now we're iComputing!

It’s been a lot of fun. OS X is built on Unix, and you can be happy “doing it wrong” by hanging out in the ugly and gross command line underworld. I found myself wanting to keep learning and playing with my new toy at home over the weekend. I missed it. It’s a beautiful machine, and I love to learn and explore new things. I suddenly realized I was looking at the price of new iMacs. I kinda want one for home, too.

Steve’s revenge!

Scott Carpenter was first introduced to programming in 1981 when Scott’s dad, an electrical engineer, gave him his first computer to encourage his interest in programming. It worked – in 1998 Scott graduated from St. Cloud State University with his Bachelor of Science in business computer information systems. Scott went on to work for Cargill for 15 years as a software developer where he focused on application integration, integration design and development. In 2013, Scott’s journey brought him to The Nerdery where he puts his technical expertise to use as a PHP, WordPress and Java developer.

Tech Tip: Receive NFC Data in an Adobe AIR Android App

Andrew Watson is a developer here at The Nerdery and one of our many Android OS black belts. With so many projects coming through The Nerdery, our developers invariably run across problems that will require some lengthy research. The following article is a result of research done in conjunction with a project that utilizes NFC (Near Field Communication) sensor technology. NFC is is now being included in specific smart phones. The most popular applications of NFC include mobile wallets, and data sharing between two NFC-enabled devices.

Receive NFC Data in an Adobe AIR Android App (by Andrew Watson)

Getting NFC data into an Adobe AIR for Android application is much easier than it might seem. At least, it’s a lot easier than I thought it would be.

I’m currently working on an app that has just such a need. Specifically, users will run the AIR app and then swipe their device over a physical display that has some NFC tag hotspots. The app needs to receive the NFC data from each tag and react in a specific fashion. I didn’t know much about NFC data when I began working on this current app, and since AIR doesn’t have official support for NFC I figured I would need a native extension to properly handle things.

It turns out it is *much* simpler than needing to implement any sort of native functionality. All you have to do is add the proper intent handling to the “manifestAdditions” portion of the AIR application xml. I’ll put a sample xml block here, and then explain it:

<manifestAdditions>
<![CDATA[
    <manifest android:installLocation="auto">
        <!-- I left out the "default" AIR manifest additions that are there by default. You should keep those. -->

        <uses-permission android:name="android.permission.NFC" />
        <uses-feature android:name="android.hardware.nfc" android:required="true" />

        <application android:enabled="true">
        	<activity android:excludeFromRecents="false">
        		<intent-filter>
        			<action android:name="android.intent.action.MAIN"/>
        			<category android:name="android.intent.category.LAUNCHER"/>
        		</intent-filter>

        		<intent-filter>
        		    <action android:name="android.nfc.action.NDEF_DISCOVERED" />
        		    <category android:name="android.intent.category.DEFAULT" />
        		    <data android:scheme="{SCHEME}" android:host="{HOST}" />
        		</intent-filter>
        	</activity>
        </application>
    </manifest>
]]>
</manifestAdditions>

The key to making everything work is in effect “extending” the <application> node – which prior to discovering this I didn’t know was possible. By adding the appropriate NFC intent filter information – the “default” way you handle NFC data in Android – the AIR application is handed the day the same way any other Android application would be.

In the xml sample above, I have  my AIR app responding to a URI. I’ve got placeholders for the scheme
and host attribute values. Most commonly these will represent a web URL; in
that case scheme would be “http” and host would be the domain – “yourdomain.com”. But if you want to use proprietary data, you can do that too. Scheme and host can be anything. Scheme could be “my” and host could be “uri.value” which would represent “my://uri.value.” It doesn’t really matter, the data will make it to the AIR app. This is just one way of targeting NFC data into an app – there are a number of different ways to use intent filters in Android to target varied NFC data types. Virtually all of them should work as manifest extensions for an AIR app. The Android documentation has information about targeting NFC data..

Extending the manifest only gets us so far; it doesn’t tell us how the data gets into the AIR application. Thankfully, Adobe has facilitated that too. Anytime the app responds to intent filter data, an InvokeEvent.INVOKE event gets fired. Check out the below class:

package
{
    import flash.desktop.NativeApplication;
    import flash.events.InvokeEvent;

    public class NFCDataHandling extends Sprite
    {
        /**
        * Constructor
        */
        public function NFCDataHandling()
        {
            NativeApplication.nativeApplication.addEventListener(InvokeEvent.INVOKE, onApplicationInvoke, false, 0, true);
        }

        protected function onApplicationInvoke(event:InvokeEvent):void
        {
            //If there is any intent data that was passed to the app - in this case, say an NFC URI - there
            //will be data in event.arguments. It will be an array, and the first item will be
            //the full URI string, i.e. {SCHEME}://{HOST}/{ARGS}
        }
    }
}

To be clear, the invoke event will in fact be fired when the app starts up, but will *also* be fired whenever the NFC intent data qualifies and is passed to the app. This means that the app will be handed any and all relevant NFC data and can respond by reading the events.arguments data.

This also has the added effect of launching your AIR application whenever the relevant NFC data is swiped. That’s actually what Android generally means to have happen with NFC data – the user swipes their device and the relevant specific application is opened. This will in fact do that. But as this post discusses, it will also pass in the NFC data while the application is running.

If you aren’t wanting the app to be launched on NFC data but instead *only* respond to the NFC data while the application is open, that requires more complexity. It does, in fact require a native extension interacting with something called the “Foreground Dispatch”. A very intelligent person has actually cracked the code and has made, well, the source code available for free! You can see it here:

http://code.google.com/p/ane-lab/source/browse/#svn%2Ftrunk%2Fmobile%2Fandroid%2Fjava%2Fnfc-foreground-dispatch

I hope this helps!

Andrew Watson is a 2002 graduate of the University of Minnesota, with his Bachelor of Science in computer sciences. Andrew went to work for Kroll Ontrack as a .NET developer before joining The Nerdery in 2008 as a programmer/analyst. An expert in a variety of technologies such as .NET/C#, Android app development, Javascript and Actionscript 3, Andrew has significantly contributed to projects for clients such as Microsoft, Cisco and HTC. In 2010, in recognition for his work and leadership, Andrew was promoted to senior developer. Not content to stop there, Andrew frequently puts his made-for-radio voice to work as The Nerdery’s unofficial voice-over go-to guy.

Not paying doesn’t necessarily make you ‘the product’

Derek Powazek pokes some holes in the old “If you’re not paying for the product, you are the product” adage.

“The ‘you are the product’ line is most often repeated when a company that provides a free service does something that people don’t like. See Instagram’s recent terms change or any Facebook design update. The subtext is, this company does not serve you, you don’t pay for it, so shut up already.

But that’s crazy talk.”

Google Doodle Celebrates 1st Computer Programmer, Ada Lovelace

The Washington Post has a nice write-up about the woman behind the Doodle.

2012 KPCB Internet Trends Year-End Update

Snagged from Daring Fireball who said “slides 9 and 22 seem apt. Slide 24 is probably the best.”

Kill the password

Okay, two Wired articles in a row. Sorry, but they’re super interesting this week. Today we’re pointing you in the direction of Kill the Password. In this one Mat Honan, whose digital life was erased by hackers this summer, talks about passwords and how they can’t keep us safe in a world of interconnected accounts and easy to Google personal information.

“The common weakness in these hacks is the password. It’s an artifact from a time when our computers were not hyper-connected. Today, nothing you do, no precaution you take, no long or random string of characters can stop a truly dedicated and devious individual from cracking your account. The age of the password has come to an end; we just haven’t realized it yet.”

The Patent Problem

If you have some free time today take a deep dive into Wired’s The Patent Problem which takes a good long look at the history of the US Patent system, how software came along and kind of messed things up, and why some people say it’s strangling innovation. It’s a long one, but interesting nonetheless.

The past three decades of wanton patent-granting have created a disastrous environment for innovation. Today it’s practically impossible to build anything without violating a patent of some kind—and risking a multimillion-dollar lawsuit for your troubles. Once intended to protect lone inventors, patents now form a kind of shadow tech industry, in which billions of dollars are spent on amassing huge portfolios. (A recent New York Times article noted that Apple and Google, companies that define themselves by innovation, now invest more in patent acquisition and defense than in research and development.)

Bonus, when you read the whole article you can learn about Patent Troll (which is a thing I didn’t know existed).