After nearly 60 hours of research and testing that spans back to 2013 and interviews with pro bakers, cookbook authors, a culinary instructor, and a registered dietitian, we’re confident that the Escali Primo Digital Scale is the best kitchen scale for most people. It’s among the fastest and most accurate scales we tested, and it remains powered on for longer than most others before automatically shutting off.
2005 feels like a pivotal year in my memory...went to South by Southwest for the first time. That was amazing. Not because of the event itself, but because of the people. I met, hung out with, and formed firm friendships with people whose blogs I had been reading for years—it really was like my RSS reader had come to life. It’s also where I met Cindy for the first time.
My condolences Jeremy. I know how it feels to see your feed reader come to live, it happened for me around the same time as you mention, at a different event. It brought lasting friendships like you describe your friendship with Cindy. One of those friends jokingly calls us all her ‘imaginary friends’, but there’s nothing imaginary in the life events we share, and the emotions those carry. As your evocative post about your friend Cindy shows. So, my condolences, from my RSS reader to yours.
But nothing delights the mind so much as fond and loyal friendship. What a blessing it is to have hearts that are ready and willing to receive all your secrets in safety, with whom you are less afraid to share knowledge of something than keep it to yourself, whose conversation soothes your distress, whose advice helps you make up your mind, whose cheerfulness dissolves your sorrow, whose very appearance cheers you up!
You can read On Tranquility of Mind online for free here or I enjoyed this edition from Penguin.
Remember the Anonymously-authored NY Times Op-Ed in September? Someone operating inside the administration that was keeping the White House in check with a cabal of other ‘adults’? Where is he now? Where are they now?
By far and away my favourite YouTube find of the year was All the Stations, a project of the entertainingly nerdy and passionate couple Vicki Pipe and Geoff Marshall wherein they undertake to visit every single railway station in Britain.
Perhaps the best introduction to the ethos of the project comes deep into Scotland in the video Chose Corrour, which serves as a sort of manifesto of acceptance of being interesting in interesting things. It’s okay to be a nerd.
While Geoff and Vicki completed their quest in 2017, they continue to release railway-related videos on their YouTube channel, including this week’s lovely The Shortest Train, a sequel to this summer’s The Longest Train.
Rather than traveling down the spine of Britain as they did for the summer solstice, for this winter solstice they took the 2 minute journey from Wrexham General to Wrexham Central, and did it with humour and panache.
If you fall into the All the Stations wormhole, you may not emerge for several weeks, as visiting 2,563 stations takes time. But I promise you will be delighted.
Some years ago Catherine and Oliver and I visited the Spring Geequinox while on a trip to Halifax. The event was a sort of “Comicon for people too weird for Comicon” and I was initially very, very uncomfortable with the heady mixture of Renaissance reenactors, ham radio operators, board game aficionados and furries.
But then I had a revelation: the freedom of each of those in the hall to be geeky in their own way opened up space in the universe for me to be geeky in my own way.
Printing presses and typewriters and fountain pens and bookbinding are my jam, not Dungeons & Dragons or inhabiting the soul of Shrek, but all are things that run contrary to the conventional, and the prominence and celebration of one only seeks to enliven all the others. My revelation put me immediately at ease, and that ease has taken me to places I wouldn’t have been able to get otherwise.
Visiting every railway station in Britain is, by almost any measure of reasonable, an inane thing to do. Which is why it’s such an interesting thing to do, especially when it’s done full-throated and without apology as Vicki and Geoff did it.
Bravo.
And off we go straight into part two of my summary of things that moved me in 2018!:
Raising My Shields – Year 5
Ever since the Snowden revelations in 2013 I’ve been moving more and more services I use into my own domain and made sure as much of my Internet traffic as possible is encrypted. In 2018, I’ve made a number of significant improvements. First and foremost, Nextcloud Talk has become available and I’ve been using it since its launch for end to end encrypted and self-hosted voice and video calling. While it still continues to evolve for a more ubiquitous use from mobile devices, I used it a lot for voice and video calling from PCs in 2018.
Another weak point in my communication infrastructure has been internal email. While I made sure communication encrypted between clients and the external server, emails were obviously unencrypted while on the external server. So this year I took some time to finally put an email server in place at home for family internal emails that contain things that really shouldn’t be stored outside our domain of influence.
Internet in the Sky – And Elsewhere
Again, I’ve been traveling quite a lot overseas and 2018 was the year in which Internet service was available on the majority of long distance flights I took. Not everything was well, however, as I noticed a significant slow down of the connection on some flights to the point of the system becoming almost unusable. The end of the year showed some improvement, however, as Delta Airlines showed how things can be done better. Also Internet access on board European flights started to become available and I also had a positive experience with Internet access on board a ship that took me from Sweden to Germany.
The History Track
This year, my interest in computing history had me alternating between the very small and the very big. In the last 12 months I had the fortune to be close enough to Cambridge twice to visit the Centre for Computing History and also to walk through the city to see where the British microcomputer revolution of the 1980s happened with Sinclair, Acorn and ARM. After reading a number of books about assembler programming in the 1960s, punch cards, Fortran and paper tape, I came across a book about how the Lyons teahouse company developed the ‘Leo’ computers in the 1950s and 60s. An unbelievable story. Bigger still is the story about Seymor Cray and supercomputing. After reading ‘The Supermen‘ I found out that the Living Computer Museum in Seattle has the first supercomputer, a CDC 6500, still running and allows anyone with an interest to log in remotely via SSH to see how supercomputing started in the 1960s.
A lot has been written about the history of Silicon Valley and Leslie Berlin’s book ‘The Troublemakers‘ tells the story from an angle of people how made a difference but are less well known today by the general public. The origin of the Internet has also been a topic for me in the past and I continued to find more interesting information about the last 20 years here and from a German perspective here. And finally, I gave my first talk about computing history this year at the Vintage Computer Festival in Berlin about mobile computing in the past 30 years and wireless network evolution during that time. In addition to that I’ve put together a list of the mobile devices I owned since I last wrote about this topic so many years ago in 2009.
And last but not least I re-discovered ‘Paradroid‘ and found ‘Die Welt am Draht‘, a distant German ancestor of ‘The Matrix’.
A New Smartphone With LineageOS
One thing that deserves a section of its own is that I could finally move to a new smartphone. As I like my privacy and detest being exploited and tracked by Google and others, I ran CyanogenMod and later LineageOS on my Samsung Galaxy S4 and S5 for quite a number of years. But the devices were aging and I was elated when LineageOS became available for the Samsung S9, the latest of the Galaxy series in 2018. Better than ever, I also discovered that I didn’t have to install a single Google app.
Miscelaneous
And finally there have been many smaller tidbits that should also not be forgotten. While I’ve gotten rid of Windows in my household entirely some years ago, I do have to work with the OS from time to time. However, the Windows Subsystem for Linux I explored this year (see here and here) will offer interesting new possibilities in the future for remote administration, support and backup. Speaking about administration, a talk during this year’s ‘Gulaschprogrammiernacht’ in Karlsruhe made me explore tmux for the first time. By now I am sure there’s not a single hour when I sit in front of my notebook I don’t make good use of its functionalities to multplex shell sessions in a single window. And last, but not least I discovered ‘Mastodon‘ that I like a lot more than Twitter and I’ve moved my source code from Github to Gitlab after the former was bought by Microsoft.
Again a year full of twists and turns and I’m very much looking forward to what 2019 will bring! Happy new year, ya’ all!
I spent the day puttering around the house and reading. I’m on book 3 of an endless series of historical fiction by Cynthia Harrod-Eagles, who has an amazing-sounding name and a love of describing angst and drama that spans many generations. Book 1 started out in the 14th century during the Wars of the Roses. Book 2 was mostly Henry VIII. And Book 3 is mostly Queen Elizabeth, well, not mostly about her but the family the books are about, the fictional Morlands from … somewhere near York, I think, are sometimes at court. So it’s against a backdrop of Queen Elizabeth and Mary Queen of Scots and so on.
This has led me to lots of Wikipedia reading – like I wanted to know all about the guy who married Mary Queen of Scots and his unhappy ending (syphillis maybe but also murdered and THEN blown up as well by some barrels of gunpowder) and then the unhappy ending of the other guy who married her (maybe abducted and raped her, then married her?) (thrown in prison but I’ve forgotten the exact events – but he was chained to a column in a Dutch prison for 10 years and died there.)
These gruesome deaths led me, though, to strike gold:
Mary Queen of Scots’ embroidered badge of a ginger cat wearing a crown playing with a mouse, embellished with her initials MA intertwined to look like a rune. Maybe the cat was her cousin Elizabeth and she was the mouse!
Whatever it may have meant, it’s delightful!!! I think it would make a really neat replica embroidered badge! Someone should get it manufactured and sell them on Etsy!
The books can be a little traumatic (I was squicked by the amount of 14 year olds who marry old guys) and there is like, constant weird incest and trauma, and children who are super charming and beloved and then DIE DIE DIE or maybe everyone dies, and I got really sad about Anne Boleyn, but they aren’t like “The Kingdom of Little Wounds” level of trauma and that has got to be the #1 gross book ever for its multiple times we get a syphilitic Queen’s very public ob/gyn exams described; keep in mind I absolutely love that book but I have to warn people when I recommend it that it goes deep.) I continue reading them since I am very curious how Harrod-Eagles is going to sustain this strange family all the way into World War II. I’m also admiring the volume of her output, I mean, from her Wikipedia page it sounds like she works full time and writes these doorstop epic novels on the weekend and there are like, 30 of them. So impressive. Anyway, I am getting just slightly INTO the vibe of these books. Is there someone sympathetic? Is there someone they absolutely shouldn’t shack up with, like, their secret half brother or their actual uncle or their husband’s brother or a hoydenish Scottish girl who rides into battle and hates men, or, their paralyzed and sickly first cousin, or like, a travelling homosexual actor or a bloodthirsty Earl who may have murdered his first and second wives? In Morland logic this is like catnip to a catte. The more inappropriate the match the faster they freaking leap into bed, get INSTA-PREGNANT and then their children and grandchildren accidentally do it all over again.
Meanwhile, a list of things I have glued today: my thumb to danny’s glasses frame; the glasses frame to itself; a large flowerpot in 6 pieces; a small bone china horse. I had to soak my thumb and the glasses in nail polish remover for several minutes before I was able to scrape my thumb free of the glasses with a knife.
And now, for something completely different from what I usually write about: As a sort of Summer resolution, I decided that I was going to do something else with my dwindling free time, and one of the things I hit upon was getting deeper into music making. The other two (before you ask) are getting back to grips with 3D animation on Blender 2.8 (which is now in beta) and building something in Unity (which I tinkered with over Summer to get a feel for).
You can probably guess that the reason I’m writing about this a few days before Christmas is that it took me a while to have the time to do anything about it. Surprisingly, music was the one that actually took, and it’s been taking up a lot more time of late.
All the above have something important in common–i.e., a combination of circumstances that made me gradually dip my toes into want to do something beyond technology instead of just tinkering with it more. My day job as an Azure advisor/architect demands a lot in terms of keeping up with tech, but that has always been (comparatively) easy for me.
What hasn’t been easy is not owning the end product and doing almost zero creative work (and I don’t mean design thinking sessions)–i.e., no coding, no product deliverables, and, most importantly, nothing I can call my own.
After all, work is only rewarding when you have purpose behind it, and despite some gratifying happenstances, I needed to spend some time doing something that was entirely under my own control and that entailed creative work.
Since my first “real job” was doing 3D animation and I’ve been tinkering with Blender and Unity for years, I briefly toyed with the idea of diving into both in earnest (like many folk, I’d love to code a game of my own for the sheer craftsmanship of it), but I wanted something that I could do anywhere given that I am (too) often on the road.
Comparatively, music has much less overhead–you don’t need sophisticated hardware, only time and inspiration. Music is also instantly gratifying–something either works great or not at all, which also appeals to my need for continuous experimentation.
But, in truth, I blame the OP-1, which I mentioned a few weeks back. Well, not the OP-1 specifically, but the sheer brilliance of it. I spent a long time going over (literally) years of YouTube videos of people playing the OP-1 ever since Teenage Engineering launched it, and the things people can do with it (and its genius tape recorder metaphor) are utterly surreal.
I would have loved to get an OP-1, but since it’s essentially made of unobtanium these days (no recent production runs, speculation that the product was terminated, etc.) and that I find the newer OP-Z rather less enticing, I decided that I might as well make better use of all the stuff I already had, as well as the vast swathes of computing power I carry about with me.
Besides a lovely (and recently repaired) Clavinova piano, I also had a Kawai K1-II whose ROMs were shot and which I kept going for a few years using a MIDI adapter and some ingenuity. The Kawaii took up too much space, so I eventually gave it away to a friend (who has been trying to resurrect it) and bought a three-octave Korg microKEY Bluetooth keyboard:
It is great bang for the buck (the keyboard is velocity-sensitive and has a nice feel to it), and Bluetooth support means I can use it on iOS with zero hassle. But what software was I going to use?
Known in the trade as DAWs, these integrated suites seem to be undergoing a resurgence these days on iOS as it starts to become a viable “pro” music platform. There are a bunch of new entrants in this space for multiple platforms (FL Studio, Nanostudio 2, etc.), but the main contenders I remembered on the Mac are still around these days:
I’ve been playing around with Garageband since day one, and there is around zero motivation for my using anything else on the Mac until this gets upgraded to the rank of a serious hobby, but since I wanted to do stuff on iOS as well, I started looking further afield.
The first thing I hit upon was Caustic 3, which runs pretty much everywhere (Mac, Windows, Android and iOS). One of the reasons I was drawn to it was happens to be free (and unsupported) on Windows and macOS, and given that I usually have an Android phone with me, it had the added appeal of letling me play around without expending critical battery life on my own phone.
In case you want to try it out, be forewarned that Caustic is extremely rough on macOS (it doesn’t even have an app icon), but works pretty well, and has the added appeal of letting you build an entire virtual rack’s worth of different synths:
I like it a lot, but the iOS version hasn’t been updated in a while and I find the loop/pattern approach to be a bit of a pain.
However, the deal breaker for me is that the retro vibe that pervades Caustic (and the effort required to make it cross-platform) also means that it integrates poorly with iOS, and that moving things across machines is a low-level affair via a built-in FTP server (truth be told that it works, but I would much rather have it work with iCLoud and the Files app).
This was a recurring theme throughout my adventures, and one that has led me to a rather polarizing take on things: In the long run, I’m going to try doing music on iOS first, and use the Mac as a fallback whenever things are far enough along to “upgrade”.
And, overall, it doesn’t get any better with Garageband (with one exception I’ll get into in a follow-up post).
However, I find it terribly frustrating that Garageband on the Mac can’t write back to iOS projects properly (you can mix down everything and save an iOS project, but you can’t edit the original tracks), and that iOS insists on displaying Mac projects but can’t actually open them.
You can start something on iOS and continue working on another iOS device (depending on whether or not you have the right samples and AU plugins), but you can’t really work on the same file as on a Mac, even if you’re only using standard instruments.
Digging into things a bit, I found that both Logic and Garageband, to my horror, seem to record absolute paths for assets into files, which is the kind of thing that really ought to have been fixed ages ago.
I suspect third-party DAWs have it easier here, but I’m not going to invest on a matching set of DAWs for both platforms… At least not yet. But I did spend a fair bit hunting down other software.
The big takeaway from scouring the iOS music landscape over the past few months is that there is a lot of software out there in various states of maturity or neglect. Most of the modern apps I came across support Bluetooth MIDI and the iOS Files app in one form or another, but they tend to be the exception–a lot of the cooler stuff is way older, with some apps having been on the App Store for over two years without any updates.
Case in point: I came across a brilliant sampling application called Samplr that worked fine on an older iPad running iOS 10, but which simply couldn’t record audio under iOS 12. I think it was the first time ever that I used the App Store refund option, and it was a pity, since one of my ongoing quests is to find something that can mimic the OP-1 workflow, and it was the closest thing so far.
In a nutshell, the overall state of application integration and the hoops developers have to jump through to do inter-app data exchange on iOS mean that to move non-MIDI stuff around, you’re still largely reliant on copy/pasting via AudioCopy/AudioShare or re-routing audio through Audiobus and doing lots of switching between apps, which makes for a pretty lousy workflow.
Audio Unit plugins are much better in that regard (they run inside the host application and manipulate MIDI data), but judging from the amount of Audiobus-compatible applications, I’m likely in the minority here.
But I did find some amazing things out there, many of which were free (fortunately, because this looks like a great way to spend a substantial amount of money on very enticing apps).
Let’s start off with the Mac first, since I spent a good while investigating my options there. While looking at Logic, I hit upon Main Stage, which is focused on live performances and automation:
However, the hidden gem here is that for a moderate price (around €30), it provided me with an absolutely gigantic amount of high-quality instruments and audio samples I can use from Garageband, since all the Logic family apps share the same core components and asset libraries.
In fact, the Logic/Garageband ecosystem has a lot going for it, and there were two other great finds that resulted from my monkeying about in it. The first was the Logic Remote iOS application, which turned my old iPad into a useful control surface:
The second was a bit of a nostalgia trip, for way back when I got the Kawai K1-II I always thought I would eventually buy a Yamaha DX7 to replace it. That never happened, but after all these years I was able to find Dexed, which is entirely free and can be used both standalone and as an Audio Unit in Garageband:
Amazingly enough, it can even import original sysex patches (of which there are a few thousands floating about on the Internet), so in between this and everything that comes with Main Stage, I don’t expect to be needing much else on the Mac.
On a similar vein, I came across FM Player while searching for a DX7-like synth for iOS. FM Player isn’t a fully-fledged synth (it merely plays back sampled presets), but Synth One is, and both can be used in Garageband via inter-app audio (which is a bit limiting since you can only record the audio and not the MIDI events, but workable):
I’ve also been using ROLI’s Noise application, which works as a full Audio Unit plug-in inside Garageband, popping up inside its UI and playing nicely with MIDI.
Right now, the biggest annoyance I have is that I can’t keep more than a couple of audio applications open on my iPad mini 4, which struggles mightily to run either Garageband and Noise or any other Audiobus-linked pair of applications.
Also, in this age of wireless headphones, I’ve had mixed results with Bluetooth audio–headphones seem to have very low latency, but I tried connecting my iPad to an Anker SoundCore 2, and there was quite noticeable lag (which I am positive had nothing to do with my Korg keyboard, which was also connected via Bluetooth MIDI).
I am temporarily placing the burden of blame on the SoundCore, since a similar experiment with my Mac (where I can tune the audio buffers) had even worse results. But a bigger problem is that older applications like Caustic, insist on using the headphone jack or speakers, and I have to route them through Audiobus (again at the expense of considerable CPU load on my iPad mini).
From a more creative perspective, I am also somewhat frustrated with Garageband‘s built in sampler, which works but is hardly remarkable (or flexible when it comes to editing the audio itself). I am looking for an alternative to Samplr, but nothing seems to fit, and there seem to be no takes on Teenage Engineering’s workflow on the OP-1 (which is kind of sad, since I see Teenage Engineering as the Apple of synthesizers at this point).
I understand both Cubasis (which is the iOS version of Cubase) and NanoStudio have much nicer samplers, but I’m not sure I want to play with yet another DAW (althogh to be fair Cubasis seems to have rather a large following).
Still, I’m having fun, and will soon have new toys to play with in this realm. I only wish I had a lot more time, and that I had paid more attention to music theory classes…
I was tired of fixing Microsoft Account sync settings and just moved on. Exported all the Edge bookmarks on Surface Pro and imported them on Lenovo Yoga.
I have to be pragmatic here. There aren't many people who run five Windows PCs in parallel. For now I am moving the heavy loads from Surface Pro to Yoga C930, so I will be able to swap Surface Pro the second week of January. If you want to continue looking at new machines, you have to stay nimble and move on. I don't believe in unbox, test a few days, and then review. I want to expose myself to a machine for a much longer period.
This Yoga C930 deserves a good ride. Yeeehah!
This morning I read Tyler Cowen's conversation with Paul Romer. At one point, Romer talks about being introduced to C.S. Peirce, who had deep insights into "abstraction and how we use abstraction to communicate" (a topic Romer and Cowen discuss earlier in the interview). Romer is clearly enamored with Peirce's work, but he's also fascinated by the fact that, after a long career thinking about a set of topics, he could stumble upon a trove of ideas that he didn't even know existed:
... one of the joys of reading -- that's not a novel -- but one of the joys of reading, and to me slightly frightening thing, is that there's so much out there, and that a hundred years later, you can discover somebody who has so many things to say that can be helpful for somebody like me trying to understand, how do we use abstraction? How do we communicate clearly?
But the joy of scholarship -- I think it's a joy of maybe any life in the modern world -- that through reading, we can get access to the thoughts of another person, and then you can sample from the thoughts that are most relevant to you or that are the most powerful in some sense.
This process, he says, is the foundation for how we transmit knowledge within a culture and across time. It's how we grow and share our understanding of the world. This is a source of great joy for scholars and, really, for anyone who can read. It's why so many people love books.
Romer's interest in Peirce calls to mind my own fascination with his work. As Romer notes, Peirce had a "much more sophisticated sense about how science proceeds than the positivist sort of machine that people describe". I discovered Peirce through an epistemology course in graduate school. His pragmatic view of knowledge, along with William James's views, greatly influenced how I thought about knowledge. That, in turn, redefined the trajectory by which I approached my research in knowledge-based systems and AI. Peirce and James helped me make sense of how people use knowledge, and how computer programs might.
So I feel a great kinship with Romer in his discovery of Peirce, and the joy he finds in scholarship.
Ole Zorn has just discounted Pythonista 3 to $4.99 over the holidays, which means you have zero excuses not to get what is (by far) the best self-hosted iOS development environment out there (for years now).
I cannot praise it enough (my kids started out coding in it, and love the app to the point where they have already delved deep into the native UI bindings), and I love the way it keeps getting substantial improvements.
On the other hand Editorial is lagging behind (it doesn’t even have iCloud Files support), and looks set to remain that way given that Pythonista brings in 50x more revenue.
Nevertheless, here’s hoping for Python 3.7 on the next Pythonista, and even deeper integration with Shortcuts.
Phishing is [still] the primary way attackers either commit a primary criminal act (i.e. phish a target to, say, install ransomware) or is the initial vehicle used to gain a foothold in an organization so they can perform other criminal operations to achieve some goal. As such, security teams, vendors and active members of the cybersecurity community work diligently to neutralize phishing campaigns as quickly as possible.
One popular community tool/resource in this pursuit is PhishTank which is a collaborative clearing house for data and information about phishing on the Internet. Also, PhishTank provides an open API for developers and researchers to integrate anti-phishing data into their applications at no charge.
While the PhishTank API is useful for real-time anti-phishing operations the data is also useful for security researchers as we work to understand the ebb, flow and evolution of these attacks. One avenue of research is to track the various features associated with phishing campaigns which include (amongst many other elements) network (internet) location of the phishing site, industry being targeted, domain names being used, what type of sites are being cloned/copied and a feature we’ll be looking at in this post: what percentage of new phishing sites use SSL encryption and — of these — which type of SSL certificates are “en vogue”.
Phishing sites are increasingly using and relying on SSL certificates because we in the information security industry spent a decade instructing the general internet surfing population to trust sites with the green lock icon near the location bar. Initially, phishers worked to compromise existing, encryption-enabled web properties to install phishing sites/pages since they could leech off of the “trusted” status of the associated SSL certificates. However, the advent of services like Let’s Encrypt have made it possible for attacker to setup their own phishing domains that look legitimate to current-generation internet browsers and prey upon the decade’s old “trust the lock icon” mantra that most internet users still believe. We’ll table that path of discussion (since it’s fraught with peril if you don’t support the internet-do-gooder-consequences-be-darned cabal’s personal agendas) and just focus on how to work with PhishTank data in R and take a look at the most prevalent SSL certs used in the past week (you can extend the provided example to go back as far as you like provided the phishing sites are still online).
You can use the aquarium package [GL|GH] to gain access to the data provided by PhishTank’s API (you need to sign up for access and put you API key into the PHISHTANK_API_KEY environment variable which is best done via your ~/.Renviron file).
Let’s setup all the packages we’ll need and cache a current copy of the PhishTank data. The package forces you to utilize your own caching strategy since it doesn’t make sense for it to decide that for you. I’d suggest either using the time-stamped approach below or using some type of database system (or, say, Apache Drill) to actually manage the data.
Here are the packages we’ll need:
library(psl) # git[la|hu]b/hrbrmstr/psl
library(curlparse) # git[la|hu]b/hrbrmstr/curlparse
library(aquarium) # git[la|hu]b/hrbrmstr/aquarium
library(gt) # github/rstudio/gt
library(furrr)
library(stringi)
library(openssl)
library(tidyverse)
NOTE: The psl and curlparse packages are optional. Windows users will find it difficult to get them working and it may be easier to review the functions provided by the urlparse package and substitute equivalents for the domain() and apex_domain() functions used below. Now, we get a copy of the current PhishTank dataset & cache it:
if (!file.exists("~/Data/2018-12-23-fishtank.rds")) {
xdf <- pt_read_db()
saveRDS(xdf, "~/Data/2018-12-23-fishtank.rds")
} else {
xdf <- readRDS("~/Data/2018-12-23-fishtank.rds")
}
Let’s take a look:
glimpse(xdf)
## Observations: 16,446
## Variables: 9
## $ phish_id <chr> "5884184", "5884138", "5884136", "5884135", ...
## $ url <chr> "http://internetbanking-bancointer.com.br/lo...
## $ phish_detail_url <chr> "http://www.phishtank.com/phish_detail.php?p...
## $ submission_time <dttm> 2018-12-22 20:45:09, 2018-12-22 18:40:24, 2...
## $ verified <chr> "yes", "yes", "yes", "yes", "yes", "yes", "y...
## $ verification_time <dttm> 2018-12-22 20:45:52, 2018-12-22 21:26:49, 2...
## $ online <chr> "yes", "yes", "yes", "yes", "yes", "yes", "y...
## $ details <list> [<209.132.252.7, 209.132.252.0/24, 7296 468...
## $ target <chr> "Other", "Other", "Other", "PayPal", "Other"...
The data is really straightforward. We have unique ids for each site/campaign the URL of the site along with a URL to extra descriptive info PhishTank has on the site/campaign. We also know when the site was submitted/discovered and other details, such as the network/internet space the site is in:
glimpse(xdf$details[1])
## List of 1
## $ :'data.frame': 1 obs. of 6 variables:
## ..$ ip_address : chr "209.132.252.7"
## ..$ cidr_block : chr "209.132.252.0/24"
## ..$ announcing_network: chr "7296 468"
## ..$ rir : chr "arin"
## ..$ country : chr "US"
## ..$ detail_time : chr "2018-12-23T01:46:16+00:00"
We’re going to focus on recent phishing sites (in this case, ones that are less than a week old) and those that use SSL certificates:
filter(xdf, verified == "yes") %>%
filter(online == "yes") %>%
mutate(diff = as.numeric(difftime(Sys.Date(), verification_time), "days")) %>%
filter(diff <= 7) %>%
{ all_ct <<- nrow(.) ; . } %>%
filter(grepl("^https", url)) %>%
{ ssl_ct <<- nrow(.) ; . } %>%
mutate(
domain = domain(url),
apex = apex_domain(domain)
) -> recent
Let’s ee how many are using SSL:
(ssl_ct)
## [1] 383
(pct_ssl <- ssl_ct / all_ct)
## [1] 0.2919207
This percentage is lower than a recent “50% of all phishing sites use encryption” statistic going around of late. There are many reasons for the difference:
Despite the 20% deviation, 30% is still a decent percentage, and a green, “everything’s 
” icon is a still a valued prize so we shall pursue our investigation.
Now we need to retrieve all those certs. This can be a slow operation that so we’ll grab them in parallel. It’s also quite possible the “online”status above data frame glimpse is inaccurate (sites can go offline quickly) so we’ll catch certificate request failures with safely() and cache the results:
cert_dl <- purrr::safely(openssl::download_ssl_cert)
plan(multiprocess)
if (!file.exists("~/Data/recent.rds")) {
recent <- mutate(recent, cert = future_map(domain, cert_dl))
saveRDS(recent, "~/Data/recent.rds")
} else {
recent <- readRDS("~/Data/recent.rds")
}
Let see how many request failures we had:
(failed <- sum(map_lgl(recent$cert, ~is.null(.x$result))))
## [1] 25
(failed / nrow(recent))
## [1] 0.06527415
As noted in the introduction to the blog, when attackers want to use SSL for the lock icon ruse they can either try to piggyback off of legitimate domains or rely on Let’s Encrypt to help them commit crimes. Let’s see what the top p”apex” domains](https://help.github.com/articles/about-supported-custom-domains/#apex-domains) were in use in the past week:
count(recent, apex, sort = TRUE)
## # A tibble: 255 x 2
## apex n
## <chr> <int>
## 1 000webhostapp.com 42
## 2 google.com 17
## 3 umbler.net 8
## 4 sharepoint.com 6
## 5 com-fl.cz 5
## 6 lbcpzonasegurabeta-viabcp.com 4
## 7 windows.net 4
## 8 ashaaudio.net 3
## 9 brijprints.com 3
## 10 portaleisp.com 3
## # ... with 245 more rows
We can see that a large hosting provider (000webhostapp.com) bore a decent number of these sites, but Google Sites (which is what the full domain represented by the google.com apex domain here is usually pointing to) Microsoft SharePoint (sharepoint.com) and Microsoft forums (windows.net) are in active use as well (which is smart give the pervasive trust associated with those properties). There are 241 distinct apex domains in this 1-week set so what is the SSL cert diversity across these pages/campaigns?
We ultimately used openssl::download_ssl_cert to retrieve the SSL certs of each site that was online, so let’s get the issuer and intermediary certs from them and look at the prevalence of each. We’ll extract the fields from the issuer component returned by openssl::download_ssl_cert then just do some basic maths:
filter(recent, map_lgl(cert, ~!is.null(.x$result))) %>%
mutate(issuers = map(cert, ~map_chr(.x$result, ~.x$issuer))) %>%
mutate(
inter = map_chr(issuers, ~.x[1]), # the order is not guaranteed here but the goal of the exercise is
root = map_chr(issuers, ~.x[2]) # to get you working with the data vs build a 100% complete solution
) %>%
mutate(
inter = stri_replace_all_regex(inter, ",([[:alpha:]])+=", ";;;$1=") %>%
stri_split_fixed(";;;") %>% # there are parswers for the cert info fields but this hack is quick and works
map(stri_split_fixed, "=", 2, simplify = TRUE) %>%
map(~setNames(as.list(.x[,2]), .x[,1])) %>%
map(bind_cols),
root = stri_replace_all_regex(root, ",([[:alpha:]])+=", ";;;$1=") %>%
stri_split_fixed(";;;") %>%
map(stri_split_fixed, "=", 2, simplify = TRUE) %>%
map(~setNames(as.list(.x[,2]), .x[,1])) %>%
map(bind_cols)
) -> recent
Let’s take a look at roots:
unnest(recent, root) %>%
distinct(phish_id, apex, CN) %>%
count(CN, sort = TRUE) %>%
mutate(pct = n/sum(n)) %>%
gt::gt() %>%
gt::fmt_number("n", decimals = 0) %>%
gt::fmt_percent("pct")
| CN | n | pct |
|---|---|---|
| DST Root CA X3 | 96 | 26.82% |
| COMODO RSA Certification Authority | 93 | 25.98% |
| DigiCert Global Root G2 | 45 | 12.57% |
| Baltimore CyberTrust Root | 30 | 8.38% |
| GlobalSign | 27 | 7.54% |
| DigiCert Global Root CA | 15 | 4.19% |
| Go Daddy Root Certificate Authority – G2 | 14 | 3.91% |
| COMODO ECC Certification Authority | 11 | 3.07% |
| Actalis Authentication Root CA | 9 | 2.51% |
| GlobalSign Root CA | 4 | 1.12% |
| Amazon Root CA 1 | 3 | 0.84% |
| Let’s Encrypt Authority X3 | 3 | 0.84% |
| AddTrust External CA Root | 2 | 0.56% |
| DigiCert High Assurance EV Root CA | 2 | 0.56% |
| USERTrust RSA Certification Authority | 2 | 0.56% |
| GeoTrust Global CA | 1 | 0.28% |
| SecureTrust CA | 1 | 0.28% |
DST Root CA X3 is (wait for it) Let’s Encrypt! Now, Comodo is not far behind and indeed surpasses LE if we combine the extra-special “enhanced” versions they provide and it’s important for you to read the comments near the lines of code making assumptions about order of returned issuer information above. Now, let’s take a look at intermediaries:
unnest(recent, inter) %>%
distinct(phish_id, apex, CN) %>%
count(CN, sort = TRUE) %>%
mutate(pct = n/sum(n)) %>%
gt::gt() %>%
gt::fmt_number("n", decimals = 0) %>%
gt::fmt_percent("pct")
| CN | n | pct |
|---|---|---|
| Let’s Encrypt Authority X3 | 99 | 27.65% |
| cPanel\, Inc. Certification Authority | 75 | 20.95% |
| RapidSSL TLS RSA CA G1 | 45 | 12.57% |
| Google Internet Authority G3 | 24 | 6.70% |
| COMODO RSA Domain Validation Secure Server CA | 20 | 5.59% |
| CloudFlare Inc ECC CA-2 | 18 | 5.03% |
| Go Daddy Secure Certificate Authority – G2 | 14 | 3.91% |
| COMODO ECC Domain Validation Secure Server CA 2 | 11 | 3.07% |
| Actalis Domain Validation Server CA G1 | 9 | 2.51% |
| RapidSSL RSA CA 2018 | 9 | 2.51% |
| Microsoft IT TLS CA 1 | 6 | 1.68% |
| Microsoft IT TLS CA 5 | 6 | 1.68% |
| DigiCert SHA2 Secure Server CA | 5 | 1.40% |
| Amazon | 3 | 0.84% |
| GlobalSign CloudSSL CA – SHA256 – G3 | 2 | 0.56% |
| GTS CA 1O1 | 2 | 0.56% |
| AlphaSSL CA – SHA256 – G2 | 1 | 0.28% |
| DigiCert SHA2 Extended Validation Server CA | 1 | 0.28% |
| DigiCert SHA2 High Assurance Server CA | 1 | 0.28% |
| Don Dominio / MrDomain RSA DV CA | 1 | 0.28% |
| GlobalSign Extended Validation CA – SHA256 – G3 | 1 | 0.28% |
| GlobalSign Organization Validation CA – SHA256 – G2 | 1 | 0.28% |
| RapidSSL SHA256 CA | 1 | 0.28% |
| TrustAsia TLS RSA CA | 1 | 0.28% |
| USERTrust RSA Domain Validation Secure Server CA | 1 | 0.28% |
| NA | 1 | 0.28% |
LE is number one again! But, it’s important to note that these issuer CommonNames can roll up into a single issuing organization given just how messed up integrity and encryption capability is when it comes to web site certs, so the raw results could do with a bit of post-processing for a more complete picture (an exercise left to intrepid readers).
There are tons of avenues to explore with this data, so I hope this post whet your collective appetites sufficiently for you to dig into it, especially if you have some dowm-time coming.
Let me also take this opportunity to resissue guidance I and many others have uttered this holiday season: be super careful about what you click on, which sites you even just visit, and just how much you really trust the site, provider and entity behind the form about to enter your personal information and credit card info into.
Stratechery is taking a holiday and vacation break the weeks of December 24 and December 31. There will be no Weekly Article or Daily Updates. The Daily Update will resume on January 7.
All new subscriptions made since November 26, including during this break, will have two weeks added to their subscriptions. See you in 2019!
I find it amazing that such a simple post can generate so much comment (75 connents as of this post), but it goes to show that discourse is about both content and building an audience over time. Mostly the latter, in this case. Dan Meyer is making the point that we should distinguish between errors made by accident (such a typo) and errors made as a result of some sort of miscomprehension (such as the pattern regognition error in the example), and that we should use the term "mistake" only to refer to the first. Well, I don't know whether "the vast majority of the work we label 'mistakes' is students doing exactly what they meant to do" and I'm sure Meyer doesn't either (there's certainly no reference to a study). But more to the point, what's to be gained by not calling both of these 'mistakes'? Teachers should be correcting for both carelessness and miscomprehension, and to suggest that "Actual mistakes are worth ~0% of the class’s time / energy / wall space for posters" is to misunderstand that education is about more than just concepts and cognition.
Web: [Direct Link] [This Post]
This is a few days old but I want to make sure this item does not get lost in the shuffle. The idea here is that people have a natural 'set point' that they drift back to - new information, for example, is not likely to change one's core beliefs, which is (partially) why you cannot simply appeal to reason and evidence to (say) persuade a person to quit smoking. The purpose of misinformation, suggests Mike Caulfield, is to shift this set point in both individuals and society, so that we redefine what's reasonable and expected. Gradually, over time, trust in media or elections can be undermined,not by a single article, but through a constant barrage of background noise that gradually hifts our trust levels. This concept arises in other areas - the idea of the Overton Window, for example, or the propaganda technique of the Big Lie.
Web: [Direct Link] [This Post]Arjen Kamphuis went missing in August, and there’s still no clue as to what might have happened to him. All scenarios are open (even if those aren’t the scenarios feverish twitterati still keep dreaming up, replacing sparse facts with dopamine oozing speculation): an accident, someone did something to him, or a self chosen disappearance. His friend Ancilla van de Leest writes evocatively about what it means to his close ones to not know. The mix of hope, heart break, guilt and optimism against the odds. Because for each of the scenarios there’s a way to rationalise it as what might have happened. Yet, they all carry their own form of deep pain, and should Arjan return, by now it will not be the same Arjan that went missing. There’s a road forwards, but it’s never a road back to the spot you came from. That point in time is forever unreachable in the available evolutionary space, whatever may follow.
As Ancilla writes, for every scenario there are clues that potentially match. One scenario is self disappearance, and she points to what the combination of an active brain, stress, and a bit of isolation can do. It’s one part of why I find her posting so evocative. I know what it’s like to have your brain on fire, and how the notion emerges to just lose oneself, walking off into the mountains on my own. At times people have known how I was, but precisely at such potential inflection points they didn’t. I would never have allowed them to know.
I know Arjen since over a decade. His work on open government, open source and then increasingly privacy and cyber security, regularly overlaps in terms of events, network connections and content with some of the work I do. So we bump into each other. On a long late train ride to Enschede with us the only two passengers in the entire compartment, a good time ago, we discussed the state of the world and our work for governments. I told him I am an optimist. Arjen said I had to be, as I am a father. He’s right. Fatherhood is a reason I’m optimistic. An older reason however is that it’s a survival tactic from decades ago, back when it was tempting to disappear. It’s not a choice really. I have to.
That’s the second reason Ancilla’s words resonate with me, likely also because she recently became a parent too. Where she writes that whatever your perception of the state of the world, commit to keeping the ones you hold dear close, to ensure you know how they feel and vice versa. It’s a path out of the struggle with conflicting emotions she describes. Out of the struggle, by embracing the conflict. It’s not a choice really, I think. She has to.
We all have to.
Making sense, even built out of ratio, is deeply emotional.
Microsoft’s chief product officer, Panos Panay, says that the Surface line is a “core part” of how Microsoft builds products.
Panay sat down with the Independent for an interview, where he talked about the Surface, the future and Microsoft’s new position as the world’s most valuable company.
Regarding the Surface line, Panay said it will be around for the long haul. He also indicated the company was all-in on Surface.
“I think if you asked me five years ago, we were still learning,” Panay said.
“We were still trying to figure out what hardware should do to bring software to life. But now it’s not just a core part of the strategy.”
When asked about Microsoft’s failed efforts in smartphones, Panay acknowledged that Lumia was a challenge. However, he also said Microsoft learned valuable lessons from that failure, which it brought forward.
Further, Panay talked briefly about future products, although he didn’t go into specifics.
“Are we completing experiences for people at work and at home? The answer is yes. So will you see new form factors that can do that, or need to do that? The answer is absolutely.”
Finally, Panay spoke briefly about the “cool factor” that comes with being part of the world’s most valuable company.
“I’ll be honest with you, it feels good. But it’s also quite humbling,” Panay said.
“We believe we’re making a difference. We have a team and a company that is inspired. And we’re not slowing down. I feel that as a company, we’re just hitting our stride in a big way. We’ll keep pushing.”
Source: The Independent Via: Windows Central
The post Microsoft chief product officer says Surface is here for the long haul appeared first on MobileSyrup.
This paper (27 page PDF) looks at alternative methods of obtaining a consensus from a group of people (aka 'the wisdom of crowds'). While it's true that a crowd can outperform an individual, the authors suggest that dividing the crowd into small groups of five, and having the five come to a consensus, can when averaged produce better results than average produced by the large crowd. This is the basis behind a lot of 'small group' methodologies, such as the world cafe. "This result supports political theories postulating that authentic deliberation, and not simply voting, can lead to better democratic decisions," the authors write.
Web: [Direct Link] [This Post]
The main point the author wants to make is that, despite all the technology, the book hasn't really changed. "The Future Book is here and continues to evolve. You’re holding it. It’s exciting. It’s boring. It’s more important than it has ever been. But temper some of those flight-of-fancy expectations. In many ways, it’s still a potato." Sure. For those people still reading and writing text-based narratives, the book hasnt changed. But for the rest of us, things have changed a lot.
Web: [Direct Link] [This Post]
This post contains the most unusual explanation of blockchain I've seen. "That whole MARC code and the ISBN numbers on every book is managed by the U.S. Library of Congress and that ensures that no two books end up in the same spot on a bookshelf. So in essence, what I'm doing is I'm checking out the book and checking it back in without taking it for good.... So blockchain is exactly the same thing. It allows me to check in and out my transcript or other smart contracts, but I can't change it. And no two contracts or no two transcripts can end back in the same spot that they left. And therefore, it's a secure mechanism to make sure that people aren't using other people's information or data." If that explanation works for you, great, but I think it suffers the problem of explaining something complex with reference to something evn more complex.
Web: [Direct Link] [This Post]
"If we’re serious about lifelong learning and re-framing learning around the creation of new knowledge through action, it will require us to re-think our educational institutions from the ground up," writes John Hagel. "Rather than pushing content to students who are viewed as passive recipients, we’ll need to embrace a pull-based model that focuses on creating environments for people to discover and pursue their passions and helps them to connect with others who share these passions." In other words - what we've been saying here.
Web: [Direct Link] [This Post]Here’s a nice seasonal touch – a pine in Nelson Park, in the heart of the West End, decorated by, well, someone. Someone who has heart.
Rolandtj
Innovation, Science and Economic Development Canada’s (ISED) 11th annual international telecom pricing study once again confirmed that the Great White North has among the highest mobile wireless prices in the world.
ISED gathered data from from Sydney, Paris, Berlin, Rome, Tokyo, as well as Boston, Kansas City, Minneapolis and Seattle in the U.S.
The federal department broke up mobile wireless into two distinct categories.
The first took six levels into account and focused on holistic wireless plan costs, including talk, text and data.
The second category considered three levels, and focused exclusively on mobile internet.
Additionally, though ISED compared Canada’s prices to seven other countries, the department also examined the cost of wireless services within Canada.
ISED first examined wireless prices for six different levels of service.
Level 1 consisted of plans that provide 150 voice minutes, with no text or data options.
The lowest recorded price was in Toronto, where subscribers could sign up for a basic 150-minute voice plan for an average of $24.98.
The highest average price was in Winnipeg, where subcribers could sign up for a $30.24 plan.
Level 1 pricing also dropped by 16 percent between 2017 and 2018.
Level 2 provided 450 voice minutes and 300 text messages.
Prices ranged from $36.71 in Regina to $42.39 in Halifax.
Level 2 pricing decreased from $40.95 in 2017 to $39.43 in 2018.
Level 3 was categorized by plans that provide 1,200 voice minutes, 300 text messages and 1GB of data usage per month.
The lowest average price, $45.91, was recorded in Montreal, while the highest average price, $83.67, was recorded in Vancouver.
Level 3 plans were essentially unchanged between 2017 and 2018, increasing from $70.70 to $70.99.
Level 4 plans provided unlimited nationwide talk and text, as well as 2GB of data per month.
The lowest Level 4 average price was in Winnipeg, $52.77, while the highest average price was in Halifax, $86.48.
Level 4 plans decreased in price from $81.61 to $75.44, between 2017 and 2018.
Level 5 plans provided unlimited nationwide talk and text, and 5GB of data.
The average Level 5 plan cost $87.32 in 2018.
Level 6 plans were shared plans with three phone lines and unlimited nationwide talk and text, as well as between 10GB to 49GB of data.
The lowest average Level 6 plan was in Winning, $123.60, while the highest plan was in Halifax, $280.81.
Level 6 plan pricing dropped from $264.65 in 2017 to $227.87 in 2018 — a 14 percent decrease.
Canada had the second-highest Level 1 average price among the group of surveyed nations. The country’s $25.73 average was higher than the $15 global average.
Canada’s $39.49 Level 2 average price ranked second-highest and was higher than the global $22 average.
Canada’s $70.99 Level 3 average price ranked highest, and was significantly greater than the $36 global average.
Canada’s approximate $75 Level 4 average made sure that the country once again had the second-highest price, well above the $40 average.
At $87.32, Canada ranked third-highest in the Level 5 pricing competition, with the U.S.’s $98 and Japan’s $103 ranking second-highest and highest, respectively. The average Level 5 price was $55.
Finally, Canada had the highest Level 6 average price, with $223.32. The average Level 6 price was $138.
ISED also looked at the cost of roaming in the U.S., compared to the cost of U.S. subscribers roaming in Canada.
The department found that U.S. national carriers included no-cost roaming to Canada.
Unlimited U.S. plans are priced between $70-per-month and $75-per-month.
Additionally, Canadian periodic roaming plans, like EasyRoam and Roam Like Home, average out to $115-per-month, while periodic U.S. roaming plans average out to $50.50-per-month.
ISED broke down mobile internet buckets into three categories.
Level 1 ranged between 2GB and 5GB, Level 2 ranged from 5GB to 10GB, while Level 3 consisted of more than 10GB-per-month.
Level 1 pricing decreased from $43.01 in 2017 to $38.28 in 2018.
British Columbia had the lowest Level 1 average price, $37.36, while Nova Scotia had the highest, $41.96.
Regional providers were on average 18.46 percent cheaper than the incumbents.
Level 2 pricing increased slightly from $60.79 to $61.90 between 2017 and 2018.
Saskatchewan had the lowest Level 2 pricing, $37.63, while most other provinces and territories had plans in the low $60 range.
Regional providers were on average 19.04 percent cheaper.
Level 3 prices also increased slightly, from $82.28 in 2017 to $83.35 in 2018.
Saskatchewan once again had the lowest Level 3 prices, $51.98, while most other provinces and territories had prices between $83 and $85.
Regional providers were 20.08 percent cheaper.
In terms of average global speed, Canada’s Level 1 91Mbps placed the country third, behind Italy with 123Mbps and Japan with 223Mbps.
Italy’s 103Mbps ranked third in Level 2 speed, while Canada’s 104Mbps ranked second and Japan’s 175Mbps came first.
Finally, Italy’s 87Mbps ranked third, Canada’s 112Mbps ranked second and Japan’s 225Mbps ranked first.
Canada consistently had the third-highest prices in all three categories.
At $38 for Level 1, Canada placed behind the U.S.’s $43 and Japan’s $70.
An average $62 Level 2 price meant Canada ranked behind the U.S. with $65 and Japan with $75.
Finally, with an average Level 3 price of $83, Canada placed behind the U.S.’s $87 and Japan’s $106.
Source: ISED
The post ISED study confirms Canada has some of the highest wireless prices in the world appeared first on MobileSyrup.
Not to brag, but 2018 was a pretty great year for Hover, and for domain names in general. We proudly released six incredible Top-Level Domains (TLDs, or domain name extensions) that sparked and brewed new and exciting endeavours in entrepreneurship. Incredible customers, just like you, saw these new TLDs as an opportunity to build something marvelous. In 2018, we released the .APP, .CHARITY, .LLC, .PAGE, .REALESTATE, and .FAN domain extensions. Learn all about our 2018 TLD releases in our year-in-review infographic below!
We hope you have a fantastic, relaxing, and exhilarating New Year! What Top-Level Domains will we release for you in 2019? You’ll just have to wait and see. 
