Editor’s note: full disclosure — Ben is an advisor to Databricks.
I am pleased to announce a joint program between O’Reilly and Databricks to certify Spark developers. O’Reilly has long been interested in certification, and with this inaugural program, we believe we have the right combination — an ascendant framework and a partnership with the team behind the technology. The founding team of Databricks comprises members of the UC Berkeley AMPLab team that created Spark.
The certification exam will be offered at Strata events, through Databricks’ Spark Summits, and at training workshops run by Databricks and its partner companies. A variety of O’Reilly resources will accompany the certification program, including books, training days, and videos targeted at developers and companies interested in the Apache Spark ecosystem.
Offering certification in Spark reinforces O’Reilly’s commitment to help companies and developers keep pace with the latest innovations in the big data space. Over the past 18 months, Apache Spark has become the most active open source project in big data. Through June 2014, there were more than 300 contributors from more than 50 companies.
Contributions to Apache Spark vs. other open source projects in the last six months. Source: Matei Zaharia, June 2014.
Spark-related proposals for our Strata conferences have surged — Spark was a trending topic among submissions for the upcoming NYC and Barcelona events. These speaking proposals come from companies already using Spark in production and who are using Spark to solve fundamental problems. Interest in Spark Camp (a training day at Strata in NYC and Barcelona) has been strong, and we plan to offer Spark Camp at future Strata events as well.
Many of the companies I interact with are using components of the Spark ecosystem (some companies have built their entire “data stack” out of these components). In the second half of this year, the different Hadoop distributions and vendors rallied to make Spark the “standard processing engine for big data.” At the most recent Spark Summit, Databricks demonstrated how complex data applications can easily be built on top of Spark.
When I first started using Spark a few years ago, one still had to learn Scala, and the analytic libraries were still fairly limited. From the outset, I was attracted to its speed, scalability, and the fact that I could use the same programming model for a variety of problems (batch, real time, interactive, iterative). Today, the Apache Spark ecosystem has a much richer set of libraries for machine learning, graph analytics, and interactive (SQL) analysis. APIs in Python and Java have significantly broadened its user base (I have met many avid users of PySpark).
The Apache Spark ecosystem continues to grow — new libraries are announced in every release. There are now Spark meetups in cities in the U.S., Europe, and Asia. Through our publishing program and this new certification, we hope to help nurture current and future users and contributors to Spark.
If you want to learn more, or if you wish to signup for updates, please visit our Spark Certification page.
Pursuing your dreams is a big undertaking.
Well, it’s a big undertaking in theory. Actually, accomplishing any major goal is really just accomplishing a series of small goals that all lead in the same direction.
One of the best organization and time-management skills I ever learned was to break big projects into little ones.
“I can’t build this whole feature; I don’t even know where to start,” I used to think when I got a huge assignment early on in my career. I would get discouraged by how impossible the task ahead of me seemed and put off doing any work at all until the last minute because I was so overwhelmed.
Then I learned to think about big tasks as many connected small tasks. I realized I didn’t have to do everything all at once; in fact I couldn’t — it was impossible. What I had to do was take one step at a time, and keep moving on to the next little project, until the whole big one was completed. “It’s true, I can’t build the whole feature right now, but I can Google for some examples [although back then it was grepping the source code] for ideas on how to approach this problem. And I can start identifying what interfaces might exist, and I can….”
You can apply the same mindset to pursuing your dream job. When the idea of going after the next big step in your career feels too overwhelming, it can make you shut down before you even get started. But if you can break it down and turn your big goal into a list of smaller goals, it suddenly becomes simple and achievable.
“A journey of a thousand miles begins with a single step.” ― Lao Tzu
You can make progress on your dream job in 10 minutes a day.
We all have 10 minutes a day — no matter how busy we are — where we can take a moment to do something to further our career ambitions.
No one who ever achieved something great did it overnight. They did it through planning, dedication, and a lot of focus. You’ll rarely find an example of someone who achieved their dream by taking some huge random action that paid off big time. Instead, it’s slow and steady progress that wins the race.
So what can you do in 10 minutes a day to get the job you want? Here are some ideas:
(Note: not all of these have to necessarily be completed in one 10-minute session; if you want to spend 10 minutes a day for a week on one concept, for example, that’s fine. These are just meant to be examples of ways to make big progress & decisions in small doses.)
If you don’t know it already, think about what day-to-day tasks are most important to you in a job. What do you really want to do? What do you really *not* want to do? Make a mind map or other organized list of must-have’s and must-not-have’s.
Update your LinkedIn profile if it’s not in great shape. Get a current photo, and align your current role and accomplishments with the role you hope to fill next.
Start a 10-minute-a-day brand-building strategy. Subscribe to newsletters that share news about your desired field or industry. Use Hootsuite to schedule tweets, Facebook updates, and LinkedIn updates to your network sharing interesting articles you find in those newsletters.
Write down where you’d like to live, along with places where you wouldn’t mind living if the perfect job were there.
Let friends and contacts in those areas know you’re looking for new opportunities.
Consider what size team you’d like to lead or be a part of. Is your dream job at a Fortune 500 corporation or a startup? Is it a small local retailer or a global manufacturer?
Match up companies of the size you like with the locations you wouldn’t mind living in. Say you’d like to live in Seattle or Portland, and work at a company of 50 people or less. Spend 10 minutes a day researching small companies in those cities and dropping them into a spreadsheet.
Some of these companies are not a match for your interests. Eliminate the ones where you are not a good fit.
Think about how you can add value to each of the remaining companies on the list. You’ll add value in different ways to a 2-person B2B software startup than you will to a 45-person marketing firm.
Do some financial planning. Would you be willing to take a pay cut for a job you’re really passionate about? If you expect a raise, how much do you want? Do you have an emergency fund if things fall through?
If you can find the information online, research the financials of the companies you’re interested in. Recently funded companies and larger organizations are more likely to be able to provide salary bumps than newer startups.
Make a list of people you know at the remaining companies you are interested in, or people you know who may be connected to those companies.
Email one person a day from that list, asking if they’d be willing to chat about openings at the company, or just to catch up (depending on the particular person or company’s situation).
Start engaging with potential companies on social media. Follow the CEO and active employees, and connect with them by retweeting and sharing content with them.
Research networking opportunities with current employees and leaders from the company. Are they speaking at conferences? Do they hold events at their headquarters?
Tweak your resume and draft cover letters.
Ask friends and peers to review your resume and cover letters.
Submit an application a week.
Don’t let the smallness of these 10-minute achievements make them seem like they’re not worth doing. It’s quite the opposite. By taking tiny steps and making regular progress, you’ll look back in a month or a year and realize how much momentum you’ve actually built up by taking one small action every day.
This collection of small goals, all completed in the direction of one great big goal, helps you to create an incredibly powerful snowball effect of growing a more and more effective network and job search. They may feel insignificant to do one at a time — after all, what can one little tweet do? — but taking action is better than taking no action.
If you have a dream, don’t wait to start until you have more time. You’ll never have more time than you’ve got right now. So what are you waiting for? 10 minutes is all you need.
“You pile up enough tomorrows, and you’ll find you are left with nothing but a lot of empty yesterdays.” — The Music Man