RobinL

โœจย Data scientist/engineer. Blog: robinlinacre.com Twitter: https://twitter.com/RobinLinacre Lead author of Splink, for record linkage at scale: https://github.com/moj-analytical-services/splink

๐Ÿ“… Joined in 2013

๐Ÿ”ผ 1,771 Karma

โœ๏ธ 458 posts

๐ŸŒ€
15 latest posts

Load

(Replying to PARENT post)

I'm generally pro nuclear and think it should be a significant part of the energy mix. But an interesting point is that the floor price of nuclear is the cost of the turbines, which are surprisingly expensive, and aren't really getting cheaper. Solar can go much cheaper than that, potentially at least. More on this point in the recent Casey Handmer interview of Dwarkesh.
๐Ÿ‘คRobinL๐Ÿ•‘9d๐Ÿ”ผ0๐Ÿ—จ๏ธ0
๐Ÿ‘คRobinL๐Ÿ•‘1mo๐Ÿ”ผ3๐Ÿ—จ๏ธ0

(Replying to PARENT post)

I largely agree. As a counterpoint, today I delivered a significant PR that was accepted easily by the lead dev with the following approach:

1. Create a branch and vibe code a solution until it works (I'm using codex cli)

2. Open new PR and slowly write the real PR myself using the vibe code as a reference, but cross referencing against existing code.

This involved a fair few concepts that were new to me, but had precedent in the existing code. Overall I think my solution was delivered faster and of at least the same quality as if I'd written it all by hand.

I think its disrespectful to PR a solution you don't understand yourself. But this process feels similar to my previous non-AI assisted approach where I would often code spaghetti until the feature worked, and then start again and do it 'properly' once I knew the rough shape of the solution

๐Ÿ‘คRobinL๐Ÿ•‘1mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

If that were the case they'd be no need for seatbelt laws
๐Ÿ‘คRobinL๐Ÿ•‘1mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

This provides a lot of interesting info on geothermal: https://worksinprogress.co/issue/watt-lies-beneath/

And more: https://www.complexsystemspodcast.com/episodes/fracking-aust...

One interesting point made here is that the cost of turbines puts a floor price on any form.of generation which uses them, whether renewable or not, meaning in the long run solar has a big advantage: https://www.dwarkesh.com/p/casey-handmer. I don't know how accurate that is

๐Ÿ‘คRobinL๐Ÿ•‘1mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

100% agree. I've also worked as a data engineer and came to the same conclusion. I wrote up a blog which went into a bit more depth on the topic here: https://www.robinlinacre.com/recommend_sql/
๐Ÿ‘คRobinL๐Ÿ•‘1mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

May be of interest: https://www.braggoscope.com/directory (a categorised catalogue of episodes)
๐Ÿ‘คRobinL๐Ÿ•‘1mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

That stat is bonkers. China's GDP is only 5x that of UK. Total UK solar is about 19GW.

So even if you divide China's solar by 5, they added in a month what we have built in >10 years

๐Ÿ‘คRobinL๐Ÿ•‘1mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

We use aws glue for spark (but are increasingly moving towards duckdb because it's faster for our workloads and easier to test and deploy).

For Spark, glue works quite well. We use it as 'spark as a service', keeping our code as close to vanilla pyspark as possible. This leaves us free to write our code in normal python files, write our own (tested) libraries which are used in our jobs, use GitHub for version control and ci and so on

๐Ÿ‘คRobinL๐Ÿ•‘2mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

I think this may be a databricks thing? From what I've seen there's a gap between data engineers forced to use databricks and everyone else. From what I've seen, at least how it's used in practice, databricks seems to result in a mess of notebooks with poor dependency and version management.
๐Ÿ‘คRobinL๐Ÿ•‘2mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

There's a good interview with DHH, who is the creator of Ruby in Rails here: https://lexfridman.com/dhh-david-heinemeier-hansson-transcri... I have no skin in the game having never used Ruby, but I found his arguments interesting
๐Ÿ‘คRobinL๐Ÿ•‘2mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

130,000 car thefts a year. That's over ยฃ1bn loss, probably closer to ยฃ4bn. In this context the total police budget of around ยฃ20bn seems remarkably low!

You'd have thought it'd be worth insurance companies paying people to track down the thieves!

๐Ÿ‘คRobinL๐Ÿ•‘2mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

When I read the title my brain immediately jumped to a slightly different idea. With olap, I often find it annoying to figure out the joins from the fk/pk relationships, so I was imagining a tool that kind of automatically followed the links for you. A bit like how a orm gives you auto complete, but without the user having to manually enter the schema.

And I wanted it to emit the raw SQL because that's generally what I want for olap.

So I had to go at building it. If anyone's interested a very rough demo/prototype is here: https://www.robinlinacre.com/vite_live_pg_orm/

Load in the demo Northwind schema and click some tables/columns to see the generated joins

๐Ÿ‘คRobinL๐Ÿ•‘2mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0

(Replying to PARENT post)

Misleading headline:

> she would nominate Trump for the award if he was successful in getting Putin to end his war and give back all the territory his forces took from Ukraine in the conflict

So the meaning of the comment is actually the opposite of headline. It's really a criticism of the basis of the negotiation

๐Ÿ‘คRobinL๐Ÿ•‘2mo๐Ÿ”ผ0๐Ÿ—จ๏ธ0