Add Blog post on 2023 Annual Rust Survey results

[Kobzol]

This blog post contains a summary and an analysis of the 2023 Annual Rust survey. It is based on a draft prepared by @graciegregory, I have merely added charts and links to the blog post, added a few sections and modified some text.

There are some notable changes in the analysis this year:

• The blog post is actually auto-generated, and contains a bunch of JavaScript using the Plotly library, which is used for interactive charts. The scripts are located at the end of the Markdown file to make review easier, but there are still some HTML divs with charts intermingled with the text.
• The charts were improved since last years. They are now interactable, draggable and zoomable. It is possible to download them as both PNG or a SVG image. If the user doesn't have JavaScript enabled, the PNG version will be shown to them. The charts should be also more or less fine on mobile devices, but I still have to make some adjustments there.
• Questions with open answers now provide a PNG image with a wordcloud that summarizes the submitted answers.
• In addition to the blog post, there is also an accompanying PDF with a simple report that contains the charts and wordclouds for all questions of the survey.

I tried to compress the PNG images, and use SVG charts for the PDF report, to make the file sizes smaller, but all the charts and the report (and also the blog post) still take a few MiBs combined. Since we will most probably also want to include the PDF report in the surveys repo, maybe I should remove it from this PR and just link to the surveys repo directly from the blog post?

I have implemented automation for generating the report, the charts and also the blog post from data exported from SurveyHero (the survey system that we use). I expect that it should be useful for making future analyses much faster, both for the annual survey, and also for any other (e.g. micro) surveys that we do. I will send a PR to https://github.com/rust-lang/surveys with the automation scripts once I clean them up a bit.

Note that you'll probably have to render the blog post locally (cargo run) to have a chance of properly seeing the charts.

The summary doesn't talk about all the questions (or even question "areas") from the survey. However, it is still a bit longer than analyses from last years (not counting the 2022 analysis, which was very bare bones). This year, respondents can see all the data in the attached report, so I think that it is fine if we don't talk about everything. It's still possible that we left out some interesting things - you can go through the report and let me know :)

[apiraino]

Couple of comments before diving into the content:

• Can the plotly js be hosted by us instead of being served by the CDN? The whole blog post is a bit hefty (~4 mb) and takes a bit to load, due to the js. Unsure what we can do about it
• When I hover with the mouse over the graphs, the Plotly widgets render over the plot title. Seeing this on Firefox/Linux, is it only me?
  
• What are the "lasso" and "box" select widgets for? Are they useful?
• You already optimized for size the PDF report ( 👍 ), there's not much more to squeeze. But I agree with you that probably the survey repo is a better place for that
• Speaking of size, I'm curious why in just one case the SVG is bigger than the PNG:

  -rw-r--r-- 1 me me  87K Feb 12 12:25 where-do-you-live.png
  -rw-r--r-- 1 me me 241K Feb 12 12:25 where-do-you-live.svg
  

[Kobzol]

Can the plotly js be hosted by us instead of being served by the CDN? The whole blog post is a bit hefty (~4 mb) and takes a bit to load, due to the js. Unsure what we can do about it

We could serve it I guess, but that wouldn't help at all with the size of the document. We could move the custom JavaScript to an external file, which.. is actually not a bad idea, I guess. It wouldn't help with the initial load time, but it should be cached for all subsequent loads. Yeah, I'll change it to an external script.

When I hover with the mouse over the graphs, the Plotly widgets render over the plot title. Seeing this on Firefox/Linux, is it only me?

I think that's the default behavior, and it is a bit translucent, I think that it's supposed to work like this. But we can also disable the menu completely, I'm not sure if it's useful.

What are the "lasso" and "box" select widgets for? Are they useful?

I'm sure that they are for some use-cases :D Probably not here though. As I said above, we can just disable the whole menu if needed.

You already optimized for size the PDF report ( 👍 ), there's not much more to squeeze. But I agree with you that probably the survey repo is a better place for that

I can do that once rust-lang/surveys#259 is merged, so that there is a stable URL for the report that I can link to in the blog post.

Speaking of size, I'm curious why in just one case the SVG is bigger than the PNG:

Probably because there are a lot of answers in the legend of this chart, which take space in the SVG. For the PNG, the long list is cut-off at the beginning (since you can't really scroll it in the PNG), so it's smaller.

[Kobzol]

We can use https://cdn.plot.ly/plotly-basic-2.29.0.min.js, which only supports pie and bar charts (which are the two only charts that we use), which is ~300 KiB gzipped, much better than the previous 1.1 MiB size.

[Kobzol]

Added scripts to an external, switched to a smaller Plotly.js bundle and added a link to the PDF outside of this repo. The blog post now has ~40 KiB, the external JS charts ~200 KiB and the Plotly bundle ~300 KiB.

[wesleywiser]

Do we feel that the word clouds offer much additional insight? For many of them, they seem to just repeat the content offered by the survey questions (eg, "what features do you want stabilized?", "what OS do you target?", "what problems do you remember encountering?").

[Kobzol]

It depends on the question, in some cases they have some interesting answers not present in the main chart (e.g. the IDE or technology domain). They also don't take place on the blog post directly, so I don't find them obtrusive.

If you're concerned about the few KiBs they take in git, we could remove them, they are also in the PDF report (although I'm not sure how many people will open it). Or we can only keep it for selected charts.

[wesleywiser]

That's fair, the word clouds for IDE and tech domain are actually pretty nice. In general, I just find word clouds distracting and not terribly useful but if they're not actually contained in the blog that probably helps.

I'm slightly concerned that we should have some native speakers of all languages present in the word clouds look at them to make sure we're not posting anything offensive. I tried OCR for some of them and didn't see anything off hand but some of the translations didn't really make any sense, so we should make sure there aren't other idiomatic meanings to the words that appear.

[graciegregory]

Left some comments! Primarily, I've requested a slight reframing of the paragraph that mentions the DEI sub-committee.

[Kobzol]

I'm slightly concerned that we should have some native speakers of all languages present in the word clouds look at them to make sure we're not posting anything offensive. I tried OCR for some of them and didn't see anything off hand but some of the translations didn't really make any sense, so we should make sure there aren't other idiomatic meanings to the words that appear.

No need to run OCR, I have access to these words programmatically. I will try to run them through translation.

Edit: ChatGPT claims that they are fine 😆

[Kobzol]

I removed non-ASCII text from the wordclouds, that removed the weird glyphs and also made them look slightly nicer.

[Kobzol]

Charts should now be ready for prime time even on mobile phones and smaller devices.

[apiraino]

We could serve it I guess, but that wouldn't help at all with the size of the document. We could move the custom JavaScript to an external file, which.. is actually not a bad idea, I guess. It wouldn't help with the initial load time, but it should be cached for all subsequent loads. Yeah, I'll change it to an external script.

Sorry, I wasn't clear. My point of not using a CDN was that currently the blog doesn't use any content from external sources and this could be on purpose. I just want to be sure that hosting content from a CDN doesn't go against some policy 🙂

(considering the blog is already on github, I assume there is already some kind of CDN behind)

[apiraino]

apart from my comment on CDN usage, the rest of this work looks great. thanks @Kobzol !

[Kobzol]

Oh, I see. Any concerns about using a JS file from an external CDN for this blog post @Mark-Simulacrum?

[Mark-Simulacrum]

I would try to avoid it (just copying the file into our tree should be ok, presuming it has its own licensing header). Referencing a CDN we don't own seems like it might incur privacy concerns around sharing IP etc. I'm not confident either way on that but seems easy to go the other direction.

Gracie can probably help get an opinion from Foundation folks who are more familiar with relevant legislation.

[Kobzol]

Ok, inlined the script.

[apiraino]

Referencing a CDN we don't own seems like it might incur privacy concerns around sharing IP etc.

(and exactly for that reason, some people prefer to block scripts from third-party domains as they are untrusted)

[Kobzol]

This should be ready for a merge on Monday 19. 2. 2024.