Markos.fragkakis

I recently read “Hadoop – Beginner’s guide” by Garry Turkington, Packt Publishing. This is a review of the book.

As the title says, this book is intented to provide the basic knowledge required to get started in Hadoop. If you are already using Hadoop and want a more in-depth analysis, you’d rather have look at other books, such as a definitive guide or a cookbook. In addition to being an introduction to Hadoop, this book also visits the current ecosystem of Hadoop. This includes Amazon Elastic Map Reduce, Hive, Sqoop, Flume. The book is addressed to Java programmers, but a few examples are written in Ruby, so it’s a plus if you know Ruby too.

Let’s see an overview of the chapters:

Chapter 1: The first chapter gives an overview of what is Big Data and why humanity came up with it. After a discussion about scaling up vs scaling out and their advantages and disadvantages, the chapter finishes with a short history of Hadoop and Elastic Map Reduce (EMR).

Chapter 2: This chapter is a get up and running walkthrough, where the reader is guided to execute the bundled Word Count program (the Hello World of Map Reduce) on a local Hadoop cluster. Then, the same code is executed on Elastic Map Reduce.

Chapter 3: After the introductory chapters, the author explains Map Reduce (MR) and its relation to Hadoop and HDFS. The reader is then walked through writing the Word Count program by herself and again, execute it both locally and EMR.

Chapter 4: Here the reader sees slightly more advanced stuff, such as chaining MR jobs and using a distributed cache to use look-up data throughout the Hadoop cluster. Also there’s an introduction to the Streaming API, which allows easy development of MR jobs using scripting languages (examples are in Ruby).

Chapter 5: This chapter advances further with multiple input jobs, the implementation of an iterative algorithm on Map Reduce and Avro, a data persistence framework with bindings for many programming languages.

Chapter 6: The sixth chapter revolves around failure handling on a local or EMR Hadoop cluster, understanding from the log files what went wrong and how to overcome it.

Chapter 7: This chapter describes the setup of a Hadoop cluster from hardware to deciding the number of nodes, storage type, security, and HDFS.

Chapter 8: In chapter the reader is presented Hive, an abstraction layer above Hadoop. The author gives an overview of Hive-QL, the SQL-like query language and explains how the queries are dynamically translated to MR jobs, that run transparently underneath.

Chapter 9: This chapter explains how Hadoop often needs to work with the relational world, drawing data from and storing data to relational DBs. The reader can also read about Sqoop, which facilitates these operations.

Chapter 10: In chapter 10 the reader learns about Flume, a system that aggregates and moves large volumes of log data. This chapter gives set-up instructions, describes different scenarios and discusses the cases where it is an alternative to Sqoop.

Chapter 11: The final chapter of the book essentially gives directions to MR-related technologies that the reader might be interested in.
I think that key selling point of “Hadoop – Beginner’s Guide” is that it touches several peripheral technologies to Hadoop. A typical reader that wants to dive into Hadoop is most probably also interested in Elastic Map Reduce, Hive, Sqoop and Flume. So, this book is a bargain in that it combines two or more introductory books. I also appreciated the fact that the author takes care to run most examples both locally and on EMR, pointing out differences, where there are any. Finally, I found the examples sufficiently explanatory, guiding the reader through set-up instructions and programming without leaving blurry points.

The biggest disadvantage of the book is the errata, with most examples containing at least one. For someone that wants to delve into a new complicated technology such as Map Reduce, every bit of concentration that goes into finding out why the code doesn’t run is a shame. I would also like to point out a problem with the listings of tab-separated files (more than 10), where the tabs are simply not printed, making the relative examples unnecessarily difficult to understand.

The conclusion I drew is that this is a well-written beginner book on Hadoop, with the errata being it’s main problem. I would buy it if I needed a good Hadoop book tomorrow, but I would wait for the second edition otherwise. For those that bought or intend to buy the book’s current edition, you can get the latest version of the examples code and see the reported errata at the book’s official page.

To help thwart rampant insurance fraud in Russia many cars are now equipped with dash cams to capture what unfolds in front of vehicles in an attempt to aid innocent persons, law enforcement, and insurance firms. This has lead to almost unlimited hours of footage found online of unbelievable accidents, close calls, and some of the worst of human behavior. Luckily somebody took it upon themselves to edit together some of the most amazingly thoughtful actions and tender moments caught with these same dash cams and edited into this short clip. And can I just say what on Earth is up with that kid running around on the highway!? (via kottke)

Filmmaker Willie Witte is currently working on a documentary series for PBS but in his spare time he makes fun experimental films. His latest, SCREENGRAB, was made without the help of computer effects though I can’t quite figure out how. After watching this three times the hemispheres of my brain are the equivalent of cross-eyed. Music by Kevin McAlpine. (via booooooom)

The Book vase by YOY Design Studio (previously) is a house planter camouflaged as a book. Made of PMMA and PVC for water proofing the vase can be opened up to reveal the dirt inside and when closed can be inserted amongst the rest of your books to save desk space. (via fuck yeah book arts)

Salt, an essential element for all animal life, is abundant here on Earth, but it still requires extraction from stone deposits or salty waters. The process of mining that salt can produce beautiful landscapes, including deep, stable caverns, multicolored pools of water, and geometric carvings. Some of these locations have even become tourist destinations, serving as concert halls, museums, and health spas touting the benefits of halotherapy. Collected here are images of salt mines across the world, above and below ground. [31 photos]

One of the colorful brine pools that are part of a lithium salt pilot plant on the Uyuni salt lake, which holds the world's largest reserve of lithium, located at 3,656 meters (11,995 ft) above sea level in southwestern Bolivia, on November 5, 2012. (Reuters/David Mercado)

Πριν το ΕΟΠΥΥ ήμουν στο ΤΣΜΕΔΕ. Το ΤΣΜΕΔΕ ήταν ένα απαράδεκτο ταμείο άλλα τουλάχιστον είχες τη δυνατότητα να πας σε όποιον γιατρό ήθελες, να εξεταστείς και μετά να πας την απόδειξη στο ταμείο να πάρεις μέρος των χρημάτων πίσω. Κάτι που ήταν στο πλαίσιο της λογικής.

Πρόσφατα το ΤΣΜΕΔΕ (όπως και όλα τα ταμεία) εντάχθηκε στο ΕΟΠΥΥ.

Όπως και εγώ, έτσι και η κόρη μου έχει μια μικρή δυσκολία στο γράμμα ‘σ’. Εγώ δυστυχώς την κουβάλησα όλα αυτά τα χρόνια και μερικές λέξεις δυσκολεύομαι να τις πω, κάτι που μου δημιουργεί ένα μικρό τρακ πριν μιλήσω σε κοινό. Αυτό δεν ήθελα να το έχει και η κόρη μου. Έτσι αποφάσισα να την πάω σε κάποιο λογοθεραπευτικό κέντρο τώρα που είναι μικρή για να μάθει να μιλάει σωστά. Περιγράφω τη διαδικασία ακριβώς όπως είναι με απλά(;) βήματα.

1. Τηλέφωνο στο 1535. Αναμονή μερικούς μήνες.

2. Αναμονή 2-3 ώρες στο νοσοκομείο παρόλο το ραντεβού. Λαμβάνω ιατρική γνωμάτευση (πρέπει να περαστεί πρώτα από πρωτόκολλο και μητρώο, όμως. Σφραγίδες)

3. Στο τοπικό ΙΚΑ για σφραγίδα από ελεγκτή ΕΟΠΥΥ (γιατί μια σφραγίδα παραπάνω πάντα χρειάζεται)

4. Φάκελος από το λογοθεραπευτικό κέντρο ο οποίος μεταξύ άλλων περιέχει φωτοτυπίες από τα πτυχία των λογοθεραπευτών (!)

5. Κατάθεση φακέλου και σφραγισμένης γνωμάτευσης  στο ΥΠΑΔ (Αιγάλεω)

6. Μηνιαία κατάθεση αποδείξεων από το λογοθεραπευτικό κέντρο στο ΥΠΑΔ στο Αιγάλεω.

7. Δεν περιγράφω άλλο!!

Και τώρα ας απαντήσει κάποιος στην ερώτηση: Αν το κράτος δεν μπορεί να αναθέσει σε κάποια εταιρεία να αναλάβει να φτιάξει το πληροφοριακό σύστημα που θα συνδέει γιατρούς, κέντρα, νοσοκομεία, ασφαλιστικά ταμεία, φαρμακεία ώστε να μη χρειάζεται να δαπανά ο κάθε πολίτης δεκάδες ώρες σε ουρές και αναμονές και μάλιστα χωρίς κανένα όφελος για κανέναν, ας το αναθέσει σε μένα και μάλιστα δωρεάν. Ναι, αυτοπροσφέρομαι, εδώ είμαι… για σας!!

FLUIDIC is the result of a unique collaboration between Hyundai’s Advanced Design Center and Berlin-based studio WHITEvoid. The interactive light sculpture is made from 12,000 suspended spheres that act as three dimensional pixels, or voxels. Surrounded by 3D cameras the piece can sense viewer’s motions which are then translated into light patterns, but amazingly the light supplied to the individual voxels is fully external. An array of high-speed lasers project into the cloud to create the dynamic visuals in real-time. Via WHITEvoid:

A seemingly floating point cloud above a water pond and consisting of 12,000 translucent spheres marks the heart of the installation. Due to a complex computer algorithm the spheres are arranged seemingly random within the cloud. At the same time the algorithm observes the positions and projection angles of eight high-speed laser projectors that are being arranged around the artwork. They are sending out beams scanning through the arrangement of the cloud. Generating bright and dim light points, this creates a highly organic and natural distribution of voxels (3D pixels). Emerging lines and shapes finally form graphical compositions without any sweet or blind spots. Keeping the same density and intensity the FLUIDIC graphics enables their viewers to observe and interact with it from every point of view.

FLUIDIC will be on display at the Temporary Museum for New Design in Milan through April 14th.

If you liked this project, there are several other artists working with interactive light fields lately, many of which have appeared here on Colossal including the flexible Firewall, the Water Light Graffiti system and also Submergence.

Like a freak midnight rainbow, this ongoing series of lit waterfalls titled Neon Luminance is part of a collaboration between Sean Lenz and Kristoffer Abildgaard over at From the Lenz. The duo dropped high-powered Cyalume glow sticks in a variety of colors into various waterfalls in Northern California and then made exposures varying from 30 seconds to 7 minutes to capture the submerged trails of light as the sticks moved through the current. To accomplish some of the more complicated shots they strung several sticks together at once to create different patterns of illumination. For those of you concerned about pollution, the sticks (which are buoyant) were never opened and were collected at the end of each exposure, thus no toxic goo was mixed into the water. See more from the project on their website.

The folks over at Teehan+Lax have just released a new tool (you’ll need Google Chrome and a pretty kickin’ internet connection) that lets you scrape public data from Google Street View to create sweeping hyperlapse videos. What’s a hyperlapse? Via Teehan+Lax:

Hyper-lapse photography—a technique combining time-lapse and sweeping camera movements typically focused on a point-of-interest—has been a growing trend on video sites. It’s not hard to find stunning examples on Vimeo. Creating them requires precision and many hours stitching together photos taken from carefully mapped locations. We aimed at making the process simpler by using Google Street View as an aid, but quickly discovered that it could be used as the source material. It worked so well, we decided to design a very usable UI around our engine and release Google Street View Hyperlapse.

The team turned their new UI over to one of their motion designers, Jonas, who made the stunning clip above. Incredible. Some other great examples of art made with Google Street View: Address is Approximate and this clip from Giacomo Miceli. (via it’s nice that)

For almost every day last month Malaysian artist/architect Hong Yi (who often goes by the nickname Red) created a fun illustration made with common (and occasionally not so common) food. Her parameters were simple: the image had to be comprised entirely of food and the only backdrop could be a white plate. With that in mind Yi set out to create landscapes, animals, homages to pop culture, and even a multi-frame telling of the three little pigs. The project, which still appears to be ongoing, has been documented heavily around the web, but if you haven’t seen it all head over to her Facebook and read an interview on designboom. Photos will also be appearing on her Instagram at @redhongyi.

Designed by Chen Lu Wei for Megawing this fun set of four erasers lets you assume the role of barber while you work, all you have to do first is make a mistake. By using the eraser you slowly shave away the rubbery hair surface resulting in a funky new hairdo for your desktop pal, effectively turning an act of destruction into an act of creation every time you erase. Pretty sure my kid would just start with the face. May or may no be available here. (via co.design)

Swiss designer and artist Paul Grundbacher makes incredible hand-cranked marble machines that he wrote about and filmed for Matthias Wandel’s Woodgears just this weekend. Grundbacher told Woodgears that he works mostly with firewood from a local factory and that he rarely sketches anything beforehand but has the ability to fashion each piece and try it as opposed to carefully measuring things out through any sort of blueprint. All the work here spans 2009-2012 and each piece is a mixture of his own ideas and tricks learned from watching videos of other artists creating similar wooden devices. You can read more about his inspiration and methodology behind each piece right here. (via mefi)

Paul Foeckler over at LA-based Split Grain is making some pretty stunning lights and other sculptures using pieces of California cypress trees. The artist says each light can involve up to 100 hours of labor as he selects the right section of wood, slices, sands, and reassembles each piece. See several more of his recent pieces over on Etsy. (via take over time)

GoodBye Horses is a 2009 installation by artist Sandrine Pelletier at galerie Rosa Turetsky. The three galloping horses were created using suspended wool coated in black latex and tar, resulting in a stark contrast between the chaotic lines of the figures against the white gallery walls. From some angles the horses are unrecognizable, but even when brought into focus appear to be haphazard, almost violent illustrations. See much more on her website.

Two years after the the Great East Japan Earthquake of March 11, 2011, and the following tsunami and nuclear disaster, a large area around the failed Fukushima nuclear plant is still considered an exclusion zone. Namie, a small city just north of the nuclear power plant, was evacuated shortly after the quake, and its 21,000 townspeople have been unable to return since, leaving it a ghost town. At the invitation of local officials, Google recently deployed its camera-equipped vehicles to Namie to create a street view map of the deserted town so residents can see their abandoned homes, and the world can witness the remains of the disaster. On Google's Map blog, Namie's Mayor Tamotsu Baba said, "Ever since the March disaster, the rest of the world has been moving forward, and many places in Japan have started recovering. But in Namie-machi time stands still... Those of us in the older generation feel that we received this town from our forebears, and we feel great pain that we cannot pass it down to our children." I've collected some of the scenes captured by the Google Maps crew below, a glimpse into an otherworldly landscape a few kilometers north of the Fukushima nuclear plant. [33 photos]

So this is completely amazing. Destin from Smarter Every Day stopped by Orbix Hot Glass in Fort Payne, Alabama to explore a fascinating phenomenon called a Prince Rupert’s Drop. Apparently when molten hot glass is dropped in cold water it forms an object that’s almost completely impervious to brute force, even a sold hammer strike to the center of the teardrop-like shape won’t break the glass. Yet gently cut or even bump the tip of the drop and suddenly the entire thing shatters in an explosive chain reaction traveling at a speed of over 1 mile PER SECOND. Watch the video above to see the effect in 130,000 fps glory. (via the awesomer)

What!? How is this even possible? Because science, my friends. Brusspup’s (previously) latest video explores what happens when a stream of water is exposed to an audio speaker producing a loud 24hz sine wave. If I understand correctly the camera frame rate has been adjusted to the match the vibration of the air (so, 24fps) thus creating … magic zigzagging water. Or something. Here’s a little more detail:

Run the rubber hose down past the speaker so that the hose touches the speaker. Leave about 1 or 2 inches of the hose hanging past the bottom of the speaker. Secure the hose to the speaker with tape or whatever works best for you. The goal is to make sure the hose is touching the actual speaker so that when the speaker produces sound (vibrates) it will vibrate the hose.

Set up your camera and switch it to 24 fps. The higher the shutter speed the better the results. But also keep in the mind that the higher your shutter speed, the more light you need. Run an audio cable from your computer to the speaker. Set your tone generating software to 24hz and hit play. Turn on the water. Now look through the camera and watch the magic begin. If you want the water to look like it’s moving backward set the frequency to 23hz. If you want to look like it’s moving forward in slow motion set it to 25hz.

Brusspup did a similar experiment last year where it looked as if the water was flowing in reverse. Can somebody please make a water fountain that does this or would we all be deaf? (via stellar)