Vierzigundzwei

Sculpt OS for what has been working to become a general purpose operating system built off the original Genode is out with a new feature release...

It's not too common for Intel to publish new CPU microcode updates outside of their "Patch Tuesday" regiment but that happened yesterday with a Friday night release of new CPU microcode although this time is limited to the Celeron and Pentium Silver families...

Last week with the AMD EPYC 4004 review and benchmarks I tested nearly the entire product stack for these new AM5-based server processors with the EPYC 4244P (6 cores), EPYC 4344P (8 cores), EPYC 4364P (8 cores), EPYC 4464P (12 cores), EPYC 4484X (12 cores + 3D V-Cache), EPYC 4564P (16 cores), and EPYC 4584PX (16 cores + 3D V-Cache). The only EPYC 4004 class processor I wasn't able to finish testing in time was the entry-level EPYC 4124P as a 4-core processor with $149 retail price. I've now had the time to finish benchmarking that budget-focused Zen 4 server processor as well as seeing how it compares to the 4-core Skylake Xeons that were prolific for years.

While machine check exception (MCE) events tend to be uncommon, a change made by Intel engineers is accommodating the ability in the Linux kernel to store more machine check records for "when things go seriously wrong" on increasingly high core count servers...

All of the Rust feature patches have already been submitted in a pull request to Linus Torvalds ahead of the upcoming Linux 6.9 merge window...

The AMD P-State CPU frequency scaling driver for improved thermal/power/performance behavior under Linux works for Zen 2 and newer systems where the platform exposes ACPI Collaborative Processor Performance Controls (CPPC) support. There's been a caveat though of the "amd_pstate" driver having issues for the Zen2-based Ryzen Threadripper 3000 series. With a newly-published set of patches, that issue should be resolved...

Last week the AMD Inception vulnerability was made public as a speculative side channel attack affecting Zen processors and different mitigation options based on the CPU generation. There wasn't too much communication around the performance implications of mitigating Inception while over the past week I have begun benchmarking the software and microcode updates on Ryzen and EPYC processors.

Mesa's Radeon Vulkan driver "RADV" now is making use of the common synchronization framework started by Intel "ANV" Vulkan driver developers to allow for more code sharing between the drivers...

This week Intel officially launched their Xeon Scalable Processors as what they claim is "the biggest platform advancement in this decade" and will end up going head-to-head with AMD's EPYC processors...

Wine 2.0 is now officially available...

DragonFlyBSD is now the latest operating system that has decided to switch from OpenSSL to LibreSSL by default...

Version 4.9.4 of the GNU Compiler Collection is now available and it closes out the GCC 4.9 compiler series...

This month's must-haves.

When 18 strangers were thrown into a room together, Octodad was born. Now three years later, the Young Horses team is about to bring it to fruition. ...

While Fedora 20 isn't going to be released until at least December, changes are already ongoing for its successor, Fedora 21. This first major Fedora Linux release of 2014 will abandon support for quite a few older graphics processors...

by Eduardo Bravo Ortiz

“Mobile first” is the motto these days for many companies. However, being able to test a mobile app in a meaningful way is very challenging. On the Google+ team we have had our share of trial and error that has led us to successful strategies for testing mobile applications on both iOS and Android.

General

• Understand the platform. Testing on Android is not the same as testing on iOS. The testing tools and frameworks available for each platform are significantly different. (e.g., Android uses Java while iOS uses Objective-C, UI layouts are built differently on each platform, UI testing frameworks also work very differently in both platforms.)
• Stabilize your test suite and test environments. Flaky tests are worse than having no tests, because a flaky test pollutes your build health and decreases the credibility of your suite.
• Break down testing into manageable pieces. There are too many complex pieces when testing on mobile (e.g., emulator/device state, actions triggered by the OS).
• Provide a hermetic test environment for your tests. Mobile UI tests are flaky by nature; don’t add more flakiness to them by having external dependencies.
• Unit tests are the backbone of your mobile test strategy. Try to separate the app code logic from the UI as much as possible. This separation will make unit tests more granular and faster.

Android Testing

Unit Tests
Separating UI code from code logic is especially hard in Android. For example, an Activity is expected to act as a controller and view at the same time; make sure you keep this in mind when writing unit tests. Another useful recommendation is to decouple unit tests from the Android emulator, this will remove the need to build an APK and install it and your tests will run much faster. Robolectric is a perfect tool for this; it stubs the implementation of the Android platform while running tests.

Hermetic UI Tests
A hermetic UI test typically runs as a test without network calls or external dependencies. Once the tests can run in a hermetic environment, a white box testing framework like Espresso can simulate user actions on the UI and is tightly coupled to the app code. Espresso will also synchronize your tests actions with events on the UI thread, reducing flakiness. More information on Espresso is coming in a future Google Testing Blog article.



Monkey Tests
Monkey tests look for crashes and ANRs by stressing your Android application. They exercise pseudo-random events like clicks or gestures on the app under test. Monkey test results are reproducible to a certain extent; timing and latency are not completely under your control and can cause a test failure. Re-running the same monkey test against the same configuration will often reproduce these failures, though. If you run them daily against different SDKs, they are very effective at catching bugs earlier in the development cycle of a new release.

iOS Testing

Unit Tests
Unit test frameworks like OCUnit, which comes bundled with Xcode, or GTMSenTestcase are both good choices.

Hermetic UI Tests
KIF has proven to be a powerful solution for writing Objective-C UI tests. It runs in-process which allows tests to be more tightly coupled with the app under test, making the tests inherently more stable. KIF allows iOS developers to write tests using the same language as their application.

Following the same paradigm as Android UI tests, you want Objective-C tests to be hermetic. A good approach is to mock the server with pre-canned responses. Since KIF tests run in-process, responses can be built programmatically, making tests easier to maintain and more stable.

Monkey Tests
iOS has no equivalent native tool for writing monkey tests as Android does, however this type of test still adds value in iOS (e.g. we found 16 crashes in one of our recent Google+ releases). The Google+ team developed their own custom monkey testing framework, but there are also many third-party options available.

Backend Testing

A mobile testing strategy is not complete without testing the integration between server backends and mobile clients. This is especially true when the release cycles of the mobile clients and backends are very different. A replay test strategy can be very effective at preventing backends from breaking mobile clients. The theory behind this strategy is to simulate mobile clients by having a set of golden request and response files that are known to be correct. The replay test suite should then send golden requests to the backend server and assert that the response returned by the server matches the expected golden response. Since client/server responses are often not completely deterministic, you will need to utilize a diffing tool that can ignore expected differences.

To make this strategy successful you need a way to seed a repeatable data set on the backend and make all dependencies that are not relevant to your backend hermetic. Using in-memory servers with fake data or an RPC replay to external dependencies are good ways of achieving repeatable data sets and hermetic environments. Google+ mobile backend uses Guice for dependency injection, which allows us to easily swap out dependencies with fake implementations during testing and seed data fixtures.



Conclusion

Mobile app testing can be very challenging, but building a comprehensive test strategy that understands the nature of different platforms and tools is the key to success. Providing a reliable and hermetic test environment is as important as the tests you write.

Finally, make sure you prioritize your automation efforts according to your team needs. This is how we prioritize on the Google+ team:

1. Unit tests: These should be your first priority in either Android or iOS. They run fast and are less flaky than any other type of tests.
2. Backend tests: Make sure your backend doesn’t break your mobile clients. Breakages are very likely to happen when the release cycle of mobile clients and backends are different.
3. UI tests: These are slower by nature and flaky. They also take more time to write and maintain. Make sure you provide coverage for at least the critical paths of your app.
4. Monkey tests: This is the final step to complete your mobile automation strategy.

Happy mobile testing from the Google+ team.

Brainstorming is widely used by teams as a method to generate ideas and solve problems. However, many brainstorming activities are flawed and can end up hurting creativity rather than helping spur ideas. It has been well documented that traditional brainstorming, where groups come together and toss out ideas one by one, is often a flawed process. Issues with brainstorming include:

• Lack of preparation: If participants in a brainstorming session don’t understand the goals of the session in advance, they will come to the session unprepared and significant time will need to be spent ensuring all participants understand the problem before any ideation can take place.
• Limited creativity: Many brainstorming activities focus on generating unique ideas or unconnected solutions to a problem. Creativity, however, is often achieved by elaborating on individual ideas by taking them apart and improving or changing one part at a time.
• Group-think: It’s easy for groups to fixate on the first dominant idea that is expressed, reducing the production of additional ideas.
• Unequal contributions: Traditional brainstorming tends to favor outgoing, extroverted personalities. Without effective facilitation, the group may end up deferring to the most outspoken and animated participants while other participants contribute less to the discussion.
• Fear of judgement: Even with the guidance that “all ideas are valid”, participants may feel the need to “perform” and only contribute ideas that they feel will be well perceived. The fear of contributing “bad ideas” can result in a counterproductive session. Studies have shown that critique and conflict can result in better and more imaginative ideas.
• Confusing next steps: Often, teams will come up with many ideas during a brainstorming session, but struggle with what to do with those ideas.

However, brainstorming doesn’t have to be flawed. It’s possible to structure brainstorming activities to maximize the value that you can get from both individual and group thinking. The following 5 step process can help you conduct better brainstorming sessions:

1. Define Goals & State the Problem
2. Stimulate Creativity
3. Ideate Individually
4. Share, Expand, and Critique
5. Categorize and Synthesize

1. Define Goals & State the Problem

Prior to the brainstorming session (preferably 24 hours in advance), send participants a clear, succinct statement that explains the purpose of the session. State the problem that you’re trying to solve. Be cautious not to make this problem statement too specific as this can limit the amount of creative solutions to the problem. A wider, more abstract problem statement allows participants to challenge assumptions, generate new perspectives, and uncover new approaches to the problem.

Provide background materials that will help participants understand the context of the problem. Summarize these materials in a scannable format to increase the likelihood that participants will read them and at least consider the highlights. By giving participants both a summary of the background material and links to the full material, you can help them feel in control over their ability to prepare for the session without feeling overwhelmed.

While it’s important to set the expectation that participants need to prepare for the session in advance, briefly re-emphasize the goals and problem statement at the beginning of the brainstorming session to ensure all participants are in alignment and to allow for any outstanding questions to be answered.

2. Stimulate Creativity

Many brainstorming sessions ask participants to jump straight from a problem statement to solutions. Many participants may not know where to start, and will often struggle with coming up with ideas without any further direction. Use “thought triggers” to help participants think about the problem in unique ways. Here are some suggestions to draw from:

• Psychology: Create a list of psychological concepts from Stephen Anderson’s Mental Notes to guide participants to think about how psychology could be applied to the given problem.
• Perspectives: Have participants consider the problem from multiple perspectives, whether that be from your target user groups/personas or from outside perspectives.
• Parallel Worlds: Create a list of parallel worlds to help participants imagine comparisons, similarities, and differences between your problem and worlds outside of your given domain. For example, draw solutions that are inspired from fashion, television, space, the travel industry, or the construction industry.
• Highlights from research & strategy: Select a few key quotes, statistics, or findings from previous research studies or strategy documents to help participants recall inputs that may impact the given problem statement.

3. Ideate Individually 

Each participant in a brainstorming session should have an equal chance to contribute their ideas. Instead of having participants immediately shout out ideas in a group setting, allow them to generate ideas individually for a fixed amount of time. This allows you to gather everyone’s opinions and generate more ideas in a shorter amount of time.

One way to ensure you gather a diverse set of ideas and opinions is to invite a diverse group of people to the brainstorming session. While sessions shouldn’t include every single person who could have an opinion, don’t limit participation to only the design team. You can get great ideas from developers, product managers, customer service reps, etc.

Provide each participant with a stack of sticky notes or note cards and instruct them to write a single idea on each note. Aim to generate as many ideas as possible within a ~10-15 minute timeframe. The exact time you choose is dependent on the complexity of the problem you’re trying to solve. Each note could have a word, phrase, or drawing to help the participants express their ideas.

Encourage participants to not over-analyze their ideas at this stage, and to generate as many individual ideas as possible. Explain that even half-baked ideas can help the team build upon the concept.

4. Share, Expand, and Critique

Share

Once individual ideas have been generated, have each person read off their ideas one-by-one. Initially, let everyone express their ideas without discussion or critique. As each idea is read, put the idea up on a wall. You could also document ideas digitally in the form of a diagram or mind map, which could be particularly useful if you’re working with anyone remotely. If similar ideas are expressed, you can group them together, but don’t focus too much on grouping concepts until all ideas are read. This will allow you to get all ideas in front of the team so that you can more effectively discuss the ideas as a whole.

Expand

Once all ideas have been shared, begin discussing how you can build upon the existing ideas. Instead of critiquing the ideas, look to clarify them and expand upon them in order to generate additional concepts. There are a few ways in which you can facilitate this process:

• SCAMPER – Discuss how you could substitute, combine, adapt, modify, put to another use, eliminate and/or reverse an idea in order to build off the original idea and create new ideas. 
• Dissect the idea – Take an idea, and see if you can take it apart and improve or change one part of it at a time. Think about if there are any parts that make up the whole that might be able to stand on their own.
• Ask “why?” – If any assumptions come up regarding either the original problem statement or generated ideas, ask “why?” several times to help spur further discussion.

Critique

While some may argue brainstorming sessions shouldn’t include critique (initially, at least), critique is a natural extension of the idea generation activity and needs to be done in order to act upon the ideas. Once all of the ideas are up on the wall, begin critiquing the ideas. Without proper facilitation, this process can quickly become unwieldy. It’s important to structure the conversation so that each idea is thoughtfully considered and so that the group can freely add ideas and build-upon existing ideas. There are several techniques you can use to manage this process:

• Six Thinking Hats – Discuss each idea from different perspectives: just the facts (white hat), the positive aspects (yellow hat), the problems or negative aspects (black hat), emotional reactions (red hat), and alternatives building upon the initial idea (green hat). Facilitate this process (blue hat) so that you can get through the ideas quickly and don’t spend too much time on any one idea.
• Round Robin – Have each person in a group give one piece of feedback about an idea, or react to feedback from another person. Continue so that everyone has the chance to contribute their feedback.

If you want more information about critique, Adam Connor and Aaron Irizarry have put together some great information about the Art of Critique.

5. Categorize and Synthesize

After you have discussed and evaluated the group’s ideas, work to combine the ideas into categories. Aim to identify themes that connect the various concepts. You may find that you have multiple levels of groups. For example, several ideas may relate to a desired capability, and several capabilities may relate to an overarching theme. At this point, you may also remove ideas from consideration if the group agrees that they do not meet the needs of the problem.

Once you have identified these groupings, you can have further discussion over how well the generated ideas and their associated categories meet the needs of the original problem. This step is critical to making sure that you move forward with the ideas that you generate. Provide participants with evaluation criteria that help you identify how well the ideas meet the overall goals. These criteria may or may not need to be weighted depending on your particular problem and goals. Have the group rate the remaining ideas and categories against these criteria. This can be done in a few ways including:

• Rating matrix – Rank ideas on a scale (e.g. 1-5) for each of the defined evaluation criteria.
• Dot voting – Have participants select which ideas are highest priority or best meet the evaluation criteria, and vote on those ideas using a limited number of dots (with stickers, or simply checkmarks on the board of ideas).

Once your ideas and categories have been synthesized, they can be shared with your broader team and explored in terms of their feasibility and overall impact.

This method of structuring brainstorming sessions takes more time and effort than the more informal method of getting together to generate ideas as a group. However, by following a more structured method, you’re more likely to generate a larger number of ideas and a higher quality resulting set of ideas. The exact method you follow may be modified based on your team’s individual needs. If you have any other suggestions for structuring brainstorming sessions, please share them in the comments.

What can a large, established studio learn from small, independent studios? ...