Reshared post from +gwern branwen

Man, how does that work…?

"In this paper we propose stochastic depth, a training procedure that enables the seemingly contradictory setup to train short networks and obtain deep networks. We start with very deep networks but during training, for each mini-batch, randomly drop a subset of layers and bypass them with the identity function. The resulting networks are short (in expectation) during training and deep during testing. Training Residual Networks with stochastic depth is compellingly simple to implement, yet effective. We show that this approach successfully addresses the training difficulties of deep networks and complements the recent success of Residual and Highway Networks. It reduces training time substantially and improves the test errors on almost all data sets significantly (CIFAR-10, CIFAR-100, SVHN). Intriguingly, with stochastic depth we can increase the depth of residual networks even beyond 1200 layers and still yield meaningful improvements in test error (4.91%) on CIFAR-10."

I can't remember if posted about it at the time, but early last year there was a big brouhaha about a big scientific fraud. Namely, Michael J. LaCour, a grad student, faked the data for a very promising paper that supposedly showed that a short conversation with a gay canvasser could really change people's minds about gay legislation. See more about that here: http://goo.gl/JcN6cZ

Well, the scientists who figured out it was all faked did so whilst preparing their own study to look into whether this sort of "deep canvassing" worked on transgender issues.

They went ahead with their study…and it turned out even more promising than the study they debunked!

It's a rare story in science, a rare story anywhere: Where young idealists not only uncover one of the greatest frauds in recent scientific history, but then manage to validate the very idea that the fraudulent study asserted.



Using your water heater as a 12kWh battery.

The CEO of a company is sitting in his office when one of his underlings come in and tells him they've found a new policty to massively increase their profits. This new policy will harm the environment. The CEO says "I don't care about the environment, let's make some money!"

82% of people say the CEO intentionally harmed the environment.

Now, take the same scenario, wrote in the exact same way except replacing the word "harm" with "help"…only 23% of people will say the CEO intentionally helped the environment.

This is called the Knobe Effect, named for the experimental philosopher Josuha Knobe.


Happy Endings was an amazing sitcom and its early demise was a travesty.



I like counterfactuals and I'm interested in guns(*) so this is a post about a gun counterfactual.

In the US, it's common for gun supporters to say they don't support any sort of gun regulation because of the 2nd Amendment. I suspect many of these people only support the 2nd Amendment because it supports what they already want to be true. Would these people suddenly get rid of their guns tomorrow if it was discovered that without a doubt the 2nd Amendment was faked and the real text meant to drastically restrict gun ownership?

__

(*) I like shooting guns. I think they're neat pieces of machinery. I like action movies where people shoot guns (and this means by near-necessity I like action movies where people get shot by guns, though it's a little uncomfortable to admit that).

I'm also fully supportive of effective gun control measures. I'd gladly give up rights to gun ownership because it seems to me the evidence is clear enough that it would save a lot of innocent lives if done correctly.