The emotional tax of bad recommendations

tl; dr: I find most web 2.0 content emotionally draining to the point of abandoning entire platforms over it.

It wasn’t until I worked at a truly awful startup that I had ever felt (or even understood really) a state of stress so intense that I was unable to relax, not even if I tried. A state of stress for consecutive days on end.

I think we can all relate to those moments in life when you feel overwhelmed — beset on all sides by obligations or problems. In such times I try to turn to humor, distractions, and escape.

For over a year now I’ve known that certain social-media platforms I was trying to “escape” to were actually promoting stress-inducing content that was having an entirely inverse effect. “This is my problem,” I told myself, “I’m the one choosing to open these sites.”

I never even touched Twitter. Facebook was the first to go almost a decade now. Then some news sites. Then linkedIn, nextdoor. The last stragglers are reddit, youtube, and news.ycombinator.

Reddit is the simplest illustration of this phenomenon. I have an exhausting meeting coming up, I need a 10 minute breather to clear my head, I turn on reddit for cat pictures and boom suddenly I’m witnessing a cop battering a protestor. My body has an immediate visceral response. So caught up am I in righteous indignation, that I have completely forgotten in the moment about how my goal was to relax. “I need to care about this!” screams my body.

I pruned my reddit feed, and now I’m down to “r/madeMeSmile” and “r/HighQualityGifs.” Unfortunately, if I ever log out of the site I get this default feed, which is about 25% outrage-bait (e.g. a video of a group of people confronting a car driving on the sidewalk) (e.g. “Controversial law allows police to seize and sell cars of non-lawbreakers, keeping the proceeds”).

The math here is working against me. One cute picture of a cat does not negate one horrifying video of somebody slipping on the stairs and hurting themselves. Or in the case of HN, one rant about how awful interviews are traumatic and dehumanizing is not negated by one thoughtful comment.

Youtube though is the platform I’d miss the most. I seem to have no control over my youtube feed, but ideally it’d be this “Nothing political, nothing with the words ‘destroyed’/’owned’/’idiot’. Nothing about millennials/boomers. Nothing where the thumbnail is a face-palm. Nothing with the phrase ‘You wont believe'”.

I go to youtube for things like lockpicking videos, gameboy repair, primitive survival. Unfortunately, somehow I seem to get recommended a ton of Joe Rogan clips, much unwanted politics around Covid, and a mix of other neutral content. And the comments are the worst.

Unfortunately what youtube’s algorithm doesn’t understand is linking one single inflammatory bad video recommendation (even if I don’t click it) may make my entire youtube experience negative. Most topics have an emotional impact on people, and web 2.0 needs to start regulating for this, or I suspect others will eventually find themselves booting platforms wholesale too.

If this were a youtube video, this is the point where I’d be telling you that all humanity will dieeeee if you don’t freak out about what I’ve just told you. But honestly it’s not really an urgent problem. I think there’s an opportunity for us as individuals to become increasingly aware of when we’re being baited/provoked and avoid platforms. I also think there’s an opportunity for new platforms or algorithms to form that prioritize giving the user the emotional experience that they are seeking.

My proposals to reduce systemic corruption

Preface: This is written in the midst of George Floyd protests. In this moment I sense a tremendous energy for change. As somebody who has always put thought into what a better system might look like, but never had any influence, I thought I might propose some ideas now.

To my mind, the basis of all corruption is the ability for bad actors not to get punished. Fixing corruption requires minimizing situations where illegal and immoral acts can be performed by those in power without consequence. This much should be obvious, but I find it helpful to start from the basics.

I’d further break this down into two core opportunities. An opportunity to increase visibility of the behavior of those in power (e.g. body cams, reports on all rounds discharged, bystanders with cell phones, public record/count of all misconduct reports). And secondly an opportunity to ensure that such information is acted upon (after all, these protests are very much motivated by the reluctance to prosecute).

One of the challenges of oversight (e.g. internal affairs) is how neutral and objective will the third party be? Can they be bought-off? It seems there is something deep in our nature to pick a tribe and focus on its well-being. This urge is incompatible with being a servant of the people.

  • Create an environment where colluding is unsafe. More on this soon.
  • Foster a mentality that a public-servant’s tribe is the community they serve. This can be an annual oath, a pledge of allegiance, or what have you.
  • Routinely create opportunities to discover illegal behaviors

Suppose every time a police officer was asked to do an illegal favor they had to register it (via a hotline) upon force of being fired. I imagine most police would still never call this hotline. Such a system isn’t adequate. But now suppose a confederate came in and proposed an illegal activity (e.g. suppose an outsider offered a bribe to an officer). Suddenly the officer is put in a situation where (s)he must either report the behavior, or risk being fired. In theory, even a willing station chief could suss out immoral recruits this way.

Or suppose a situation is manufactured where a lone officer is asked to investigate a suspicious individual with $1,000 on his person. Does it get reported?

I tried to stay abstract, as such policies apply generally to political actors as well. There are sufficient tools to root out the dishonest, if we exercise our creativity.

The internet doesn’t need parrots

This is going to be a long one, so buckle in.

Let’s start with a hypothetical. Suppose I were to ask the average person if it was safe to apply a homemade cream everyday that had the chemical “homosalate” in it, their reaction would be “no.” Upon googling, they’d learn that the chemical homosalate has the potential to disrupt hormones. Their reaction would fall under the “common sense” aka “You’re dumb for even asking this” type of response.

At the same time, if I were to ask the average person whether it’s safer to use sunscreen or not use sunscreen, they would say “use it.” Upon googling, they might encounter an official-looking webpage from the American Academy of Dermatology saying “everyone” should use sunscreen “every day if you will be outside.”

I imagine you know what I’m going to say next: that sunscreen has homosalate in it. And it’s not just homosalate, “FDA has put the entire sunscreen industry on alert by proposing that in just two instances do we have enough safety information about ingredients to determine whether they’re safe and effective: zinc oxide and titanium dioxide.” [2] There are concerns about multiple suncreen chemicals disruption hormones.

Is suncreen why men’s sperm counts are down 50% in the last 40 years? Unlikely, but when we solve this great mystery it’s gonna turn out to be something we consider safe (because everybody else around us was doing it).

Now at this point in the article we can blame “corporations,” an easy target, change nothing, and call it a day. Or “media,” that’s a great reason to hang up one’s hat. Or maybe even “the administration,” if it’s your turn (regardless of whether this problem has persisted from a dozen administrations).

But this article isn’t about who to be mad at for 10 minutes and move on. This article isn’t sunscreen, or fertility, or the FDA, or politics. Everything you’ve read so far serves just to illustrate of a larger point.

This article is about “common sense.”

This article is about how we have a propensity to follow, and repeat “common sense.” This articles about when two pieces of “common sense” contradict, like described above, we “parrot” them both anyways. This article where I hope to convince you that we now live in an era where parroting is no longer necessary, and actively destructive.

A web of factoids

Humor me, while I describe one way to think about the way a messy tangle of opinionated people with differing experiences slowly come to a rough consensus about what’s true. What they are coming to a consensus about, and how long it takes them, varies (be it Covid, vaccines, usefulness of email, marijuana use).

I propose that most of us in society are like little machines, executing code all the time via our behaviors. Much of the code we execute is hardcoded (breathing), some of it we choose ourselves. But most of it we download from those around us, without even realizing we’re doing it.

We’re constantly watching and copying the behaviors of those around us (“norms”) following them, and then spreading them on to others. This isn’t necessarily good or bad in the abstract. Many of the things we copy are very useful (proper hygiene, how to chop vegetables). Many of the things we copy are necessary to fit in (grooming and attire). Many of the things we copy are completely arbitrary and vary between cultures (should I eat at a table rather than sitting on the floor?). Some of the things we copy are bad (e.g. for a while it was cool to smoke).

I want to emphasize this is copying and not a conscious, calculated choice. We like to pretend that we’re aware and in control of our behaviors, which is why I want to be extra clear. It’s not that people drew out a chart of “pros” and “cons” and said, “Yep, smoking does make sense for me, for the next 3 years, but eating at tables is essential.”

So the model I’m proposing is that all information dissemination can be viewed as 3 behaviors

  • Creating: Generating new information (e.g. “I invented a new type of concrete”)
  • Parroting: Relaying information blindly (e.g. “Joe just discovered a new type of concrete”)
  • Filtering: Selectively relaying information (e.g. keeping your mouth shut because you have no idea if Joe has any clue what he’s talking about)

So parroting is basically another word for the copying I’ve just described. And it’s how most information is transmitted.

Half what I “know,” I don’t actually know. For example, I knew there were 9 planets in the solar system back in 5th grade, despite having never seen any of them with my own eyes except Earth. There could have been 3, or 15, I was repeating what I was told. Later, it was decided that there are actually 8, and Pluto doesn’t count.

Most of the time this “parroting” seems practical, even indespensible. It would be pretty absurd if teachers weren’t allowed to teach how many planets there are unless they had personally verified them all. So it would seem there is a very healthy reason for us to “parrot” this information about planets (take it verbatim, pass it along without questioning or verifying it).

But books exist, and they do exactly that (parrot). And because they are printed permanently, you know nothing was lost along the way. But compared to word-of-mouth books are slow to print, expensive, and difficult to update (say, when a planet is demoted). So it would seem parroting is still useful.

But now the internet exists. And it makes parroting obsolete. We can have a widely-distributed article on exactly what planets there are, why they are planets, how we know this, photos, up-to-date information on their positions, how to verify it for yourself, and it can be updated in seconds. This is almost what wikipedia is. The value of “caching” (parroted) facts in our brains has gone down drastically now that a more reliable source can always parrot them to us through our phone.

The Parrot’s-eye view

A parrot would reject this point entirely. A parrot would say “Well there are actual agreed-upon experts out there, and there’s nothing wrong with being a relay.” And there are fields where this is fairly true.

For example, there are experts in aviation, and the fact that we can generally trust planes not to crash midflight over a track-record of decades objectively validates that those experts and systems.

But there are also fields, like economics, where the track-records of experts predicting things is scarcely better than chance, and there’s little consensus on anything, and when there is consensus it’s sometimes wrong anyways (e.g. big short).

The question of which experts are actually experts, is so confusing we almost would want an expert on the topic. What makes somebody an expert? Being on the TV box? Being in political office? Using big words? An expensive degree? A lot of followers? Having ceremonial garbs? Published journal articles?

My perspective

So then what? We can’t have authorities?

No. We can’t. You can’t get the title “authority” and then get to say whatever you want free of second-guessing. Think of it like the ideal of academia — there is no “president of science;” there are peers and slowly-earned trust. Nobody is above criticism, everybody is a peer.

We’re inevitable already heading in this direction — everybody gets an opinion, and the attention you receive is proportional to the argument you make. And when you have a good track-record, your signal gets amplified more (by people “subscribing” to you). This technique powers scientific community this is the basic premise of decentralization and “web 2.0” replacing gatekeepers.

Twitter, to some extent, is a system to replace parrots, and allow the informed to reach the uninformed without human-telephone. Though I can’t say the execution is perfect.

What should change?

I think our scientific community has long since abandoned hierarchy, and particularly this chain-of-command mentality. There is no need to parrot, rather you cite a source.

What I notice though, during covid, is how ineffective our chain-of-command government has been. I think many individuals have been weeks ahead of the government by using simply internet sources to correctly challenge a lot of “official” information around ventilators, curve-flattening, efficacy of masks, distribution of bailout money, and I expect more examples to follow.

When we become conscious of this change in truth-seeking, it will be come clear our political process will have to change with it. I believe mistakes like Vietnam would become increasingly improbably in our modern hyper-connected society.

We’ll know this succeeds when, before starting a quarantine, our elected officials publicly share risk models with the wider community, solicit input, and update decisions as on-the-ground information changes.

But if that seems like wishful thinking, let the simplest takeaway be this, that there is now a word, a derogatory one, for every time somebody plays the expert on a topic they haven’t researched by simply reiterating a buzzword. Perhaps the WHO claims masks don’t help, but you’re a parrot, you wouldn’t know (you’ve done no research), and you wouldn’t know if the WHO would know (you haven’t validated any of their “experts”).

A different github design

Earlier today HN reviewed a proposed github redesign and largely didn’t favor it.

As a senior-devops engineer, I have zero expertise and no real design credentials. Yet I think I can still do better.

See the proposed before-and-after pictures above.
Far be it from me to tell everyone else how to do their job, but here are some principles that seem intuitive to me, and maybe designers might consider them too.

  1. The visibility of an item should be proportional to its usefulness. This includes size, placement, brightness/colorfulness.
  2. There is always a flexible solution which caters to both experts and novices simultaneously.
  3. Hierarchy of UI should reflect conceptual hierarchy.

If these rules don’t click for you, then you probably have a long way to go in your UX journey. But just to spell out some of the more egregious mistakes:

  • Create new file has no business being on the same line as “Clone repo.” It is a branch-specific operation next to a repo-specific operation.
  • The current-branch dropdown/button should connected-to the file-list widget. The file-list widget is showing files of that branch. The two are logically interdependent but visually separated.
  • Wiki and Insights are features I have never used on github and may never use. They should be hidden by default. They can intelligently show for repos that have ever once used those features.


Good luck on your design journeys.

HN’s most controversial topic

Foreword:
In this piece I’m staking out an ambitious challenge in trying to unprovactively examine a topic that has thus far always proven provacative. I know in even broaching the subject, which for many is a sore topic, maybe one that they don’t believe can be resolved through sincerity anymore, I am pitting myself against both the misunderstanding and the entrenched cynicism of all past failures. Nonetheless, I have a point worth making.

 

Do you really want to read this?

Before we begin, a thought: I’ll be taking a controversial issue, one you likely have strong opinions on, and illustrating its depth. You should ask yourself if you really want to read this, considering what you’ll be losing. By accepting the nuance and grey in an issue you forfeit up the simple joy of conviction, of being the Good Side crusading against the Bad Side. If that joy is something you depend on, this piece will simply be an irritating conflict to you.

What I ask of you:

If you think you do want to read this, then prove it to myself and yourself — I ask you to say this outloud, right now, “I want to understand this issue and I’m open to considering the possibility that the people I disagree with are both intelligent and trying to make the world better.”  If you cannot even say that to yourself now, or type it, then do not bother reading on– your time is better spent investigating why you cannot sincerely proclaim neutrality.

 

Now that that’s taken care of I should finally tell you the most controversial topic of news.ycombinator. And that topic has three faces: gender, political-correctness, and truth. It encompasses Damore, but in many ways has roots that spread much further.

Two friends

I’ve interviewed two of my most respected friends — friends who are both intelligent, rational, well-intentioned, and have come to opposite sides of this issue. I expect that they can’t convince you, after all they conflict each other, but my hope is that you, as I, will find an intellectual and moral respect for them as people and finally understand how a clear-thinking person could come to such opposite conclusions.

Bill

Bill, who describes himself as “pro-Damore,” has an interesting perspective. In college he volunteered with autistic kids, and spent the better part of a semester trying to teach one young boy to say “ah,” among other developmental steps. “I only got him to do it once the whole time. It was pretty crushing. It was very clear to me that sometimes the progress is so slow it can feel pretty hopeless. I have a lot of respect for the women who volunteered there [he was the only male volunteer].”

I bring this up not because it makes him look like a stereotypical “goodguy,” but because I think it informs his larger worldview. When I ask him what his outlook is on autism he says, “Well it’s unclear. Personally, I suspect it’s increasing, even though the data is ambiguous. One in 68 having autism sounds high to me. Were 1 in 68 children autistic in the Roman days? But ultimately I think it’ll get cured, along with most conditions, through science. Probably not just in my lifetime”

My theory is that to Bill, the problems of this world, or at least one problem, is a biological fact that can be solved only through science.  When I ask him why he described himself is roughly pro-Damore he said ultimately for him he feels antagonistic to the attitudes of political correctness.

“I get that political correctness means well. But there has to be a line. Take something super controversial for example, like race and IQ. If you’re too PC to ever even do research, you’re actually going to fail at helping the world. For example, lead poisoning is known to decrease IQ and has been suspected to relate to violent crime. There are still tons of homes that have lead pipes! Even Flint, they never even got rid of the fucking lead pipes there! Because it’s allegedly too expensive. So my point with all of this is that, there may very well be certain patterns in IQ or violence in poor demographic areas, and quite possibly due to something like lead, and this poor area may also predominantly be a minority. But if we immediately assume that every time the words race and IQ come up in the same sentence that somebody’s intent is to argue racial superiority then we’ll never actually fix the biological problem– we’ll HURT the world by try to prove how sophisticated we are.”

I’m not exactly clear how this ties back to Damore, maybe it doesn’t. I ask him about it. “Well look, people said he was sexist for saying the average neuroticism across the group of women is higher than in men. I think the only important question is “is it true?” If you refuse to believe something that’s true because you wish it weren’t, you’re gonna hurt the world in the long-term. Problems get fixed with science, and science requires honesty.

When I ask him why he’s so sure about the neuroticism thing he says he read it on wikipedia. I don’t know how I feel about that, but one thing I will say for sure is that I know Bill, his opinions are coming from a place of trying to help the world.

 

Sam

I have a lot of respect for Sam, I’ve always found him to be well spoken. His clarity-of-thought counts for so much actually getting people to understand his perspective. I respect that because it’s easy to get emotional, easy to be passionate, but not so easy to run all the angles and do the due diligence on having a logically consistent perspective.

“No, I wouldn’t describe myself as a Damore supporter, but I also think that there aren’t just two sides to this issue.”

I want to pose Bill’s perspective to Sam, but it’s hard because I’m not exactly clear on Bill’s perspective. I instead ask “Why do you think think Damore was such a contentious topic?”

“Well I think there’s a fundamental disagreement on the power of words. From a legal perspective we’re all entitled to our opinions, but from a social perspective we have pretty strong norms about what can be said. And this is good, because words are actually really powerful.” I ask for clarification. “For an extreme case, what do you think would harm somebody more. Having 20 people tell the person that they are unlikable, or tripping and falling on their face?”

I see Sam has a point, but not really how it relates. I ask what the connection is. “I read Damore’s entire manifesto and it’s true he doesn’t say anything sexist. But that doesn’t mean he doesn’t violate norms. Ultimately he made a lot of people uncomfortable which matters.”

I challenge this — “Sure, but sometimes the truth is uncomfortable, surely saying things that make people uncomfortable isn’t reason in-and-of-itself for punishment. Particularly if those things are true.”

He smiles at this. “Well, yes and no. It’s true sometimes an uncomfortable truth must be said. But those who think nothing true can be offensive are kidding themselves. What if a girl at google wrote a piece suggesting male engineers are more likely to be on the autism spectrum (allegedly in support of some HR point)? It may be true, but deliberately rocking the boat like that does have real emotional effects on people and just may get punished.”

I change gears and ask him about whether political correctness can go too far, and mask issues that need correction as Bill feared. He answers, “I suppose that could happen. But I guess I personally don’t see that as a very realistic concern. At least from a broader historical perspective about historical attitudes toward Native Americans and blacks the facts actually didn’t matter that much. At the end of the day the slave owners didn’t care if their slaves were their intellectual equals, it was the social reality they subscribed to let them do what they did. It was the culture and the words. Even if the average intellect of a slave was higher than a slave owner I don’t think that would have prevented the injustices of racism.”

I ask him if he understands how somebody could see it another way, and worry that biological things like lead poisoning could be a larger problem than social things. He says, “Yes.”

Just for get a better picture, I ask him what he thinks of men’s right movements. “I think they really get the short end of the stick. I think that will change though. It’s just easy to associate a group with its most belligerent or intense members and that can put a negative face on a group– it happened with feminists in the 90’s and atheists too.”

 

 

Conclusion

Some topics become more clear as you understand them, some topics get less clear as you understand them. I think this is the latter. I cannot say how to compare the ills of social problems against scientific problems, I can’t say whether they are at odds at all.

What I can say, though, is that both Bill and Sam are good people, and I like them both.

disclaimer: I lied, both Bill and sam are actually me. All right reserved.

My Brainf Quine

A quine is something simple to describe yet surprisingly challenging to implement– a program which outputs exactly its own source-code. It is something of a rite-of-passage for an engineering afficianado. For those that consider ourselves one level beyond afficianado we always are looking to up the ante. I took two years exploratory years off after high school, and remember them fondly. Those were the days I could explore anything I wanted. Time was so abundant and problems were so scarce that I’d take on challenges like quines recreationally.

It’s a magical place to be in, when you any path feels possible and no obligations feels mandatory. It’s a time when one’s world-view is fully open, and interesting opportunities seem everywhere. It’s a time before traditional adulthood, where one can feel exhausted by unending obligations (cable bill, health insurance, change my oil, arrange my 401k, excercise more, sleep more, read more, relax more, setup dentist appointment, pickup groceries, return that item, answer those emails to those family members).

Once we’re in the “real world” it can be a challenge to remember that initial feeling of possibility. Once the lionshare of our time is spoken for, one may switch modes from expanding exploration to reduction. A mode where we filter our world into a functional place of checklists and routines to optimize staying afloat when our time, attention, and concern run short and we must ration them.

Anyways, I reminisce. But back in that era, one thing my friends and I would do is make coding challenges for each other. After a friend introduced me to “brainfuck,” a language with only 6 commands all represented as single characters, I challenged him to write a quine in brainfuck. I can see by googling that many other people like us are out there, who, like us, have been to that place where we are hungry for the next challenge to create for ourselves.

Recently I found my quine from back then it brought back memories.

 

>>>>++>+>>>>>+++>++++++>>>>>++++++>+>
>>>>++++>+>>>>>+>+>>>>>+++>++++++++++
>>>>>++>+>>>>>+++++++>+>>>>>+>+>>>>>+++>++>>>>>+++++>++++
++>>>>>++++++>+>>>>>++++>+>>>>>+++++++>+>>>>>+>++++++>>>>>
++++++>+>>>>>+>++++++>>>>>+++++++>+>>>>>++>+++>>>>>+++
>++++++>>>>>++++++>+>>>>+>+>>>>>+++>++++++++++>>>>>++>
+>>>>>++++>+>>>>>+++++++>+>>>>>+>+>>>>>+++>++>>>>>++>++
>>>>>+++>++++++>>>>>++++++>+>>>>>+>+>>>>>+
++>+++++++>>>>>++>+>>>>>++++>+>>>>>+++++++>+>>>>>+>+
>>>>>+++>+>>>>>++>+++>>>>>++++++>++>>>>>+>+>>>>>+++>+
>>>>>+>+>>>>>+++>+>>>>>++>+
+>>>>>++++>+>>>>>+++++++>+>>>>>+>++>>>>>++++++>+>>>>>++++>
+>>>>>++>++>>>>>+++>+>>>>>+>++>>>>>+++++++>+>>>>>++>+>>>>>++++++>+>>>>>+++
+>+>>>>>+>++>>>>>+++++>+>>>>>++>++>>>>>+++++++>+>>>>>+
>+++>>>>>+++++>+>>>>>++>+++++>>>>>++++++>+>>>>>++++>+>>>>>+>++>>>>>+++>+>>>>>+
+>>>>>+++>+>>>>>++>+++>>>>>+++++++>+>>>>>+>++>>>>>++++++>
+>>>>>++++>+>>>>>++>++>>>>>+++>+>>>>>+>++>>>>>+++++++>+>>>>>+>+>>>>>++++++
+>>>>>+>+>>>>>+++++>+>>>>>++>+>>>>>++++>+>>>>>+++++++>+>>>>>+>++>>>>>
+++++>+++++>>>>>++++++>+>>>>>++++>+>>>>>++>++++++>>>>>+++>+>>>
>+>++++++>>>>>+++++++>+>>>>>++>+>>>>>++++++>+>>>>>++++>+>>>>>++>++++++
>>>>>+++>+>>>>>+>++++++>>>>>+++++++>+>>>>>++>+++++++++>>>>>++
+++++>+>>>>>+>++++++>>>>>++++++>+>>>>>+>++++++>>>>>+++++++>+>>>>>++
>++++++>>>>>++++++>+>>>>>+>++>>>>>+++>++++++>>>>>++++++>+>>>>>++>
+>>>>>+++>++++++++++>>>>>+>+>>>>>++++>+>>>>>+++++++>+>>>>>++>++
>>>>>++++>+>>>>>++++++>+>>>>>++++>+>>>>>+>+>>>>>+++>++>>>>>++>+>>>>>+
+++++>+>>>>>++++>+>>>>>+>+>>>>>+++>+>>>>>+>+>>>>>+++>+++++>>>>>++++++
>+>>>>>++>+>>>>>++++>++++>>>>>+>+>>>>>++++>+>>>>>+++++++>+>>>>>
++>++>>>>>++++++>+>>>>>++++>+>>>>>+>+>>>>>+++>++>>>>>++>+>>>>>++++++>
+>>>>>++++>+>>>>>+>+>>>>>+++>+>>>>>++>+>>>>>++++++>+>>>>>++++>+
>>>>+>++>>>>>+++>+++++>>>>>++++++>+>>>>>++>+>>>>>+++>+++++++++>>>>>+>
+>>>>>++++>+>>>>>+++++++>+>>>>>++>++>>>>>++++++>+>>>>>++++>+>>
>>+>+>>>>>+++>++>>>>>++>+>>>>>+++++++>++++++>>>>>++>+>>>>>++++++>
+>>>>>++++>+>>>>>+>++>>>>>+++++>+>>>>>++>++>>>>>+++++++>+>>>>>++>+
++++>>>>>+++++++>+>>>>>>

++++++[-<++++++++++>]
<++……[-]
<<<<<<[<<<<<<]
>>>++++++
[<++++++++++>-]
<++>>++++++
[<+++++++>-]
<+>>>
[[<+<+>>-]<<
[->>+<<]>[-<<.>>]<<<.>>>>>
[-<<+<+>>>]<<[->>+<<]<[<.>-]
<<…..[->>>>>>+<<<<<<]>
[->>>>>>+<<<<<<]>>>>>>>>>]
<<<<<<[<<<<<<]>>>>>>
[<<++++++
[>++++++++++<-]>>-
[-<++>
[-<+<+++++
[>—-<-]>>
[-<++>
[-<+>
[-<<+++++
[>+++++++++<-]>>
[-<++>]
]
]
]
]
]
>[-<<.>>]
>>>>>
]

Sopping Wet — Today’s Software Ecosystem Isn’t DRY [and nobody seems to understand or care]

Tl; Dr:

  • Everyone seems to understand DRY is good at the program level, but they don’t seem to understand it at the community level.
  • Examples of useless duplication include many programming languages, libraries, package managers, data-stores, tools
  • This community duplication reduces interoperability and slows productivity across the board

Section 1: Some examples

1. Why is there more than one unix/linux package manager? Do we really need a different package manager with the same commands but renamed for each programming language? Do we really need a distinct package manager for each distro? Rhetorical question — No. We don’t.

2. Nobody seems to admit it, but Php, Ruby, Python, and Javascript are the same language, with a little sugar added here or there and different libraries.  I’m okay if not everybody wants to use curly braces but would rather indent for typing, but I’m not okay with every library for every functionality (date parsing, database connectivity, html parsing, regex, etc) being rewritten as a distinct library for every language when those languages have almost no significant differences.

This leads to a scenario where “learning a language” is more about learning the library than anything else (e.g. “How do timezones work again in PHP?”)

3. MongoDB never should have existedMongoDB should be a storage engine. The concept of a datastore that adapts its schema on-the-fly and drops relations for speed is okay, but there’s no reason the entire data-storage technology has to be reinvented to allow this. There’s no reason the entire query syntax has to be reinvented. There’s no reason the security policy has to be reinvented and all the DB drivers. There’s no reason all the tools to get visibility (sql pro) and backup the database need to be reinvented. Plus, if it were just a storage engine, migrating tables to InnoDB would be easier.

The same point holds for cassandra (which is basically mysql with sharding built in), elastic search, and even kafka (basically just WAL of mysql without columns). For example, a kafka topic could be seen as a table with the columns: offset, value. Remember storage engines can process different variations on SQL to handle any special functionality or performance characteristics as-needed.

4. Overly-specialized technologies should not exist (unless built directly around a general technology). You ever see a fancy dinner-set, where for “convenience” people are offered 5 forks and spoons, each one meant to be used slightly differently for a slightly different task? That’s how I feel about overly-specialized technologies. For example, people seem to love job queues. All job queues should be built on top of a SQL backend so that engineers get the normal benefits

  1. engineers know how to diagnose the system if it fails because it’s a common one (e.g. performance issues, permissions)
  2. engineers can always query the system to see what’s happening because it’s using a standardized query language
  3. engineers can modify the system if necessary because it provides visibility into its workings
  4. engineers can use existing backup, replication, and other technologies to store/distribute the queue (giving interoperability)

Section 2: What’s the result of all this?

  • Senior Engineers are all set back years relative to junior ones (which is bad for senior engineers, good for junior engineers)
  • The ecosystem is set back as a whole (all tools, libraries that interact with the old technology are rebuilt for the new one)
  • The company is placed in a precarious position because it now only has junior engineers in the given technology. Did I tell you that time the place I worked accidentally lost most of their customers phone numbers, because their PHP driver for mongo would convert numeric strings to numbers, and phone numbers would overflow the default integer, resulting in no fatal errors but simply negative phone numbers?
  • The company runs the risk of being saddled with a technology that will be dropped (e.g. couchdb, backbone) and will require a rewrite back to a standard technology or be perceived as behind-the-times.
  • Slow-learning / part-time engineers must keep pace with the changing landscape or face irrelevance. Those that can’t learn 10 technologies a year (a storage technology, a build tool, a package manager, a scripting language, data-monitoring tool, 2 infrastructure tools,  5 libraries, etc) will stumble.
  • Fast paced-engineers will lose half of their learning capacity on trivialities and gotchas of each technology’s idiosyncrasies (e.g. why can’t apache configs and nginx configs bare any resemblance to each other?). Once these technologies are phased out, all of that memorization is for naught. It’s a treadmill effect – engineers have to sprint (keep learning new technologies) to move forward at all, walk just to stay in place, and if you can’t keep pace with the treadmill you get thrown off the machine.

 

Section 3: The exceptions

There are a few exceptions I can think of when a complete rebuild from scratch was an improvement. One would be Git. In a few months, one of the most prominent software geniuses of our era invented a source-control system so superior to everything else that it has been adopted universally in a few years, despite the intimidating interface.

The times a rebuild is justified seem to be when many of these criteria apply:

  • You’re a known and well-respected name that people trust so much the community might standardize on what you make (e.g. Linus Torvalds, Google)
  • The existing systems are all awful in fundamental ways, not simple in easily-patchable ways. You’ve got the ability, time [and we’re talking at least a decade of support], money to dedicate yourself to this project (git, aws, gmail, jquery in 2006)
  • You can make your system backward compatible (e.g. C++ allows C, C allows assembler, Scala allows Java, many game systems and storage devices can read previous-generation media) and thus can reuse existing knowledge, libraries, and tools
  • You’re so smart and not-average that your system isn’t going to have the myriad of unanticipated flaws that most software systems you want to replace will. For example, angular, backbone, nosql, are all community fails. I theorize Go, Clojure, Haskell, Ruby, and several other high-buzz languages will evaporate.
  • Your system is already built-in-to or easily-integrated-with existing systems (e.g. JSON being interpretable in all browsers automatically, moving your service to the web where it will work cross-platform and be accessible without installation)

Section 4: What can one do?

  1. Learn the technologies that have stood the test of time: linux cli, c++/java, javascript, sql
  2. Wait years before adopting a technology in professional use for a major use-case– let other companies be the guinea pig
  3. Roll your eyes the next time somebody tells you about a new sexy technology. For whatever reason, it’s culturally “cool” to know about the “next big thing,” but professionals need to rise above such fads
  4. Next time you have a brilliant idea, instead of thinking “How great it would be if the entire dev ecosystem adapted itself to use my invention” think “Is there any open-source project out there that can be minimally adapted to accomplish my goal?”

Our obligation as leadership

The talk of importance of values is one irony of the San Francisco scene, if not human nature. The same values are discussed everywhere; so why then is it that these same values seem to be applied nowhere?

Could it simply be that it’s much easier to see the mistakes of others than our own? Perhaps those of us in positions of power often subjected to less scrutiny? Yes, to both of these. And so it becomes our own greatest personal challenge to remain true to our goals when nobody else is checking.

Let’s consider the value of ownership. What does this mean for us? It’s easy to look at the engineers who report to us and think about the times they didn’t take a personal investment in what they were doing, and how that harmed the company.

But for us to be good at our job, we must challenge ourselves to hold ownership, because it’s rare somebody will tell us when we’re not.

So what does ownership actually mean? Well, if an engineer is taking ownership of a project that to me means that she takes personal and emotional accountability for doing the best feasible job she can at it. If it’s broken on production, she is treating the lost revenue like her own.

But what does ownership look like in a manager? To me ownership is no less of an obligation. In fact we have more obligation because we have more influence. We should hold ourselves personally accountable for accurately assessing the merits of our direct reports. If a great manager screws up, he should lay awake at night until he fixes it, just as a great engineer would wake up to fix a production issue. A great manager won’t “good enough” it and wait until the next review cycle to compensate. Us not admitting to an error to save face is no less excusable than an engineer covering up when he breaks the app (which is to say absolutely inexcusable).

Ownership is about caring about the job getting done correctly, at a core level, above-and-beyond what is immediately asked of you. If you see a problem that nobody else sees, ownership is taking it up and ensuring that it gets resolved, regardless of how it reflects on you. Ownership is helping the company and the customer, even if it costs you your job (be that whistle-blowing, disagreeing with an incorrect authority, refusing to do something immoral/illegal).

If you institute an initiative that is clearly ineffective, then you should admit it and withdraw the initiative. Your self-promotion is not a contribution to the company. If you are a great manager, you won’t make it your direct reports’ job to convince you they are great; you will make it your job, your contribution to ensure everyone beneath you is being used to the best of their ability. And if your manager is great, she won’t expect you bring donuts, wear a tie, show up early, or flatter; because it will be her contribution to the company to accurately evaluate your work (and not how much she loves or hates you).

And if, when you hear this, you find it mildly irritating that anybody would ask so much of you… then don’t be surprised when those you lead act as do and not as you say. Your attitude is the irony of the SF tech scene.

Connect 4 – illustration of Matrix rotation in js

Once in an interview, I was asked to determine whether a somewhat filled in connect 4 board has a winner. Here’s an elegant solution using an arbitrary rotation function: Rotate the board 0°, 45°, 90°, 135°, and check for 4 in a row in each of those rotations.

Since I couldn’t find a good rotation solution online I decided to post a solution here (javascript):

var _ = require('underscore');

var rotateArray = function(array, degrees) {
    var radians = degrees * Math.PI / 180;
    var round = (f=>parseFloat(f.toFixed(13)));
    var coordinates = _.flatten(array.map((row,y) => row.map((value,x)=>({value,x,y}))));
    var translatedCoordinates = coordinates.map(spot=>({value: spot.value, x: round(spot.x*Math.cos(radians) - spot.y*Math.sin(radians)), y: round(spot.x*Math.sin(radians) + spot.y* Math.cos(radians)) }))
    var rows = _.chain(translatedCoordinates).groupBy(x=>x.y).values().sortBy(x=>parseFloat(x[0].y)).value();
    return rows.map(x=>_.pluck(x,"value"));
}

var hasFourInRow = function(array){
    var vectors = [0, 45, 90, 135];
    var matrixToString = (m => (m.map(r=>r.join("")).join("\n")));
    var versions = vectors.map(degrees => rotateArray(array, degrees)).map(matrixToString);
    return _.any(versions,(b=>b.indexOf("****")!==-1));
}

And to illustrate the use, here’s sample test cases:


var testCases = function(){
    var win1 = [['*','*','*','*'],
                [' ',' ',' ',' '],
                [' ',' ',' ',' '],
                [' ',' ',' ',' ']];
    var win2 = [['*',' ',' ',' '],
                ['*',' ',' ',' '],
                ['*',' ',' ',' '],
                ['*',' ',' ',' ']];
    var win3 = [['*',' ',' ',' '],
                [' ','*',' ',' '],
                [' ',' ','*',' '],
                [' ',' ',' ','*']];
    var win4 = [[' ',' ',' ','*'],
                [' ',' ','*',' '],
                [' ','*',' ',' '],
                ['*',' ',' ',' ']];
    var lose1 = [['*','*','*',' '],
                 ['*',' ','*','*'],
                 [' ','*','*','*'],
                 ['*','*',' ','*']];

    console.log([win1,win2,win3,win4, lose1].map(hasFourInRow));
    //outputs true, true, true, true, false
}

testCases();

Stories, and Meaning [at work]

Tl; Dr: All people are motivated by meaning of their work. The ability to shape the “stories” that we tell ourselves gives meaning and thus is an essential motivational tool. This meaning is also key for personal satisfaction.

Let’s consider a standard engineer. There are a lot of “outlooks” this engineer might have. I’ll go over 5 out of thousands of possibilities just to illustrate some variety:

  • I’m the star engineer here, and I’ll turn this place around. Others might not quite see it yet, but it’s only a matter of time.
  • I’m a great engineer but nobody appreciates my work. How come that other engineer got promoted, I don’t think their work is as good as mine. I should run the company, the CEO is only CEO because he has blue-blood 1% investor friends. The world is rigged against me.
  • I’m so excited to be an engineer. I earn multiple times the median household income and I get perks too, while my other friends are still paying off their college debt.
  • Work’s good. It’s just a job though.
  • Why am I always the one under pressure? Nobody ever thanks me no matter how hard I work. Why do I get mistreated? I’m such a victim. It’s not right.

What I’m proposing is that the same engineer having the same experiences could retell that same experience in any of the ways I listed above.

In fact, you’ll find that a lot of people gravitate towards common narratives. I have a friend, let’s call him Fry who always sees the same story: The big guy is mistreating the little guy. One thing I learned quickly was that he liked and actively chooses (if not entirely consciously) to see the world in these terms. And if I want his help, I can motivate him by giving him a rebel story he can play a part in.

For example, if I’m trying to create a new tool, I could tell it in a way that plays up the opposition, especially if it’s an entrenched “senior” group that’s cautious about new ideas, and it’ll make him want to help.

Everybody has a narrative or two that resonates with them. I have another coworker, Bender, who always promotes the story: People are stupid. I know not to try to change his story, because it’s a choice and likely inspired by some very frustrating life experience (I imagine those frustrations are valid, albeit rather long-lived). If I need him on a PR I can ask it in a way that implies I need his expertise. That way he act can out his fantasy of undoing the damage of incompetent/indifferent through the act of improving my code. A win/win.

Sometimes these narratives are just plain counterproductive. I can think of experiences when teammates were annoyed that some “undeserving” party “stole credit.” In these situations you can try to rewrite the narrative, though it takes a certain skill. One way to accomplish this is to bring up an opposing narrative like, “Yeah so his name was mentioned in a meeting big deal… I don’t do great work so some manager can mention my name, I do it because there are literally millions of users who are experiencing what I make and I think I’m lucky to be able to be in a position to help so many people. That’s something I can be proud of.” [And as an aside their is no duplicity in this. It is our choice whether to aim to “do good” or to accrue material. Each philosophy has its upsides.]

And the fact that people are choosing to live these stories is an important fact. For a long time, I would have thought doing this type of thing was “manipulative” or “tricking” people into work when I could instead explain to them why their underlying narrative was too black&white. But knowing that these narratives are choices, with symbolic importance, that give meaning to the individual who holds them, I see now that people choose to live these stories hoping to “play them out.”

And most importantly, we all have narratives. Mastering other peoples narratives is a great tool, but mastering your own is probably more important.

———

Part 2.

If you’re a leader in an organization, you need to be aware of the stories that occupy the minds you oversee. Do people see themselves as battling each other for recognition? Do they see themselves as allies against a great evil? Do they see themselves as victims of your rule?

People within the company all have their narratives, and those attitudes are contagious. Your actions will have a great role in deciding which attitude wins out.

I’ve seen organizations torn apart by leaders who were out-of-touch with the effects of their actions. The cultural effects you have are of supreme importance. Actions like requiring engineers start an hour earlier, for example, are incredibly dangerous, because they give room for the narrative “We are seen as code monkeys” to thrive.

To win this battle you must understand the narratives that you are battling against. You must truly understand them, you must know and appreciate the day-to-day of your workers. For example, you must know that your Office Manager Lela is frustrated, sleep-deprived, is wondering where her life is going, and is starting to feel like the “manager” in her title is meaningless. When you know the people you work with you’ll know what stories are compelling to them. When you understand, you will be capable of offering the service of arming them with a better story, a brighter story, a more compelling story. Because as bad as Lela’s life is, it’s better for everyone if she feels like her work matters at the end of the day.

This is your value.