Automate KubeCon event listings #53519
Conversation
✅ Pull request preview available for checkingBuilt without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
lmktfy
left a comment
There was a problem hiding this comment.
This is good, but there's a challenge: how does this interact with localization?
- eg: make sure to mark the text as left to right in case the parent page isn't
- eg: we might want to use a localized date format
- other ideas can fit too
/area web-development
| """Scrape KubeCon events from Linux Foundation calendar""" | ||
|
|
||
| headers = { | ||
| 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36' |
Thanks for the feedback! Here's what we can do:
other suggestions are welcome too |
singh1203
left a comment
There was a problem hiding this comment.
LGTM!
@Caesarsage, Thank you. This is a solid implementation that eliminates the need for manual event updates. The filtering logic and error handling are particularly well done.
| # Configuration | ||
| EVENT_URL = "https://events.linuxfoundation.org/about/calendar/?_sf_s=kubecon" | ||
| EVENT_LIMIT = 2 | ||
| OUTPUT_PATH = "data/kubecon_events.yaml" |
There was a problem hiding this comment.
Non-blocking: This could also use CLI flags for URL/limit/output to help with local testing, but the current constant-based approach is also reasonable for a small maintenance script.
|
This idea seems interesting :)
region name - does it mean country name and city name ? for example in: there is "North America" + "Atlanta". |
|
I'd say that the region is "North America". KubeCon North America was most recently held in the city of Atlanta. |
|
Either before or after we merge this, we could ask the CNCF for a more machine readable version of the page we scrape. Maybe there is a WordPress plugin that can add RDFa data to the event feed or something? Or they can serve the data we're after as jCalendar? |
Yes, i agree with this |
How do one get this informations from CNCF |
|
To contact the CNCF, we use https://github.com/cncf/servicedesk @kubernetes/sig-docs-leads can send a message on your / our behalf @Caesarsage (and it's OK to propose that!) |
nate-double-u
left a comment
There was a problem hiding this comment.
Thanks for this @Caesarsage
A concern: this may circumvent the steering committee's ability to approve site-wide changes. Usually any update to the banner needs to be approved. Tho, I suppose if this is run and generates a PR that could be approved that would alleviate the concern.
Also, each banner has a UTM link associated with it generated by the LF Marketing team that wouldn't be captured by scraping the event page.
How would this manage scheduling the banners (and ensure there is no overlap between existing banners, or if there is, how would this decide which has precedence)?
|
There are several places we mention KubeCon. This PR is about mentioning it on https://k8s.io/ (and localized equivalents) We also have site wide banners, that this PR doesn't plan to touch. For an Urchin / Google Analytics string, we could put that into a Netlify variable and inject it into production builds. The string isn't secret but putting it into a variable helps avoid tainting the metrics with forks and stale site. |
The proposed change here is a utility function that can be used to generate the data used by the shortcode for the event list section on the https://k8.io/ . So yes someone will need to generate the data with the function and raise the PR for the changes. And as Tim mentioned it doesn't tough the site wide banner . |
|
Thanks for clarifying the context, I had it in my head this was a banner tool. |
Hello @Caesarsage and @lmktfy, thank you for proposing this automation approach. Regarding localization, my understanding is that the current discussion leans toward registering only the region names in i18n, such as North America, Europe, India, and similar, and making those translatable via Hugo i18n. Is that correct? If so, locations like Amsterdam or Mumbai would not be localized and would always be displayed as their original English names. I just wanted to confirm whether this matches your intention. Thanks in advance for the clarification. |
I’m not sure if my previous question was understood correctly, but I was trying to refer to exactly the same thing :) |
|
Yes @dkarczmarski, thanks. I’m aware this point has already been raised by you :), |
|
Hi @Caesarsage, |
Yes @seokho-son |
538ca43 to
12ec04b
Compare
499a09b to
b6c34a6
Compare
|
|
||
| [kubectl_ref_view_full] | ||
| other = "Zobacz pełną dokumentację" | ||
|
|
There was a problem hiding this comment.
the change at line 290 seems unnecessary :)
There was a problem hiding this comment.
Agreed, we should probably not change localizations in this PR.
nate-double-u
left a comment
There was a problem hiding this comment.
We may be able to build an API to call to get the same data in a cleaner way. Could you add the spec you want as an issue to the Events site here: https://github.com/linuxfoundation/lfevents/issues/new
Otherwise, in the meantime, this looks good to me, thanks @Caesarsage.
/lgtm
|
|
||
| [kubectl_ref_view_full] | ||
| other = "Zobacz pełną dokumentację" | ||
|
|
There was a problem hiding this comment.
Agreed, we should probably not change localizations in this PR.
|
|
||
| # Configuration | ||
| CALENDAR_URL = "https://events.linuxfoundation.org/about/calendar/?_sf_s=kubecon" | ||
| EVENT_LIMIT = 2 |
There was a problem hiding this comment.
nit: why 2? If there are 3 events coming up this would skip one.
There was a problem hiding this comment.
I guess if we really want to show 3 upcoming events, change this locally and run the script with that setting.
|
LGTM label has been added. DetailsGit tree hash: d055d7ddeeb4935a896af6d63185e0450a8f256d |
| {{ range $data.events -}} | ||
| {{ $startDate := time.AsTime .start_date -}} | ||
| {{ $endDate := time.AsTime .end_date -}} | ||
| {{ $regionKeys := dict "North America" "kubecon_region_north_america" "Europe" "kubecon_region_europe" "India" "kubecon_region_india" "Japan" "kubecon_region_japan" "China" "kubecon_region_china" "Event" "kubecon_region_event" }} |
There was a problem hiding this comment.
nit: could put this mapping in (eg) data/events/region_name_heuristics.yaml
lmktfy
left a comment
There was a problem hiding this comment.
/lgtm
I don't see harm from merging this as-is.
| REGIONS = { | ||
| "North America": "North America", | ||
| "Europe": "Europe", | ||
| "India": "India", | ||
| "Japan": "Japan", | ||
| "China": "China", | ||
| } |
| """Check if title matches the expected KubeCon event pattern.""" | ||
| if KUBECON_TITLE_MATCH_RE.match(title): | ||
| return True | ||
| print(f" Skipped (not a main KubeCon event): {title}") |
There was a problem hiding this comment.
(nit)
| print(f" Skipped (not a main KubeCon event): {title}") | |
| print(f" Skipped (not a main KubeCon event): {title}", file=sys.stderr) |
| soup = BeautifulSoup(response.text, 'html.parser') | ||
| ld_event = _extract_json_ld_event(soup) | ||
| if not ld_event: | ||
| print(f" Warning: No JSON-LD Event found at {url}") |
There was a problem hiding this comment.
(nit)
| print(f" Warning: No JSON-LD Event found at {url}") | |
| print(f" Warning: No JSON-LD Event found at {url}", file=sys.stderr) |
| print("\nNo matching KubeCon events found!") | ||
| print(" Looking for: 'KubeCon + CloudNativeCon <region>'") |
| print(f"\nNetwork Error: {e}") | ||
| print(" Could not fetch the events page.") |
| print(" Could not fetch the events page.") | ||
| exit(1) | ||
| except Exception as e: | ||
| print(f"\nError: {e}") |
There was a problem hiding this comment.
| print(f"\nError: {e}") | |
| print(f"\nError: {e}", file=sys.stderr) |
|
LGTM label has been added. DetailsGit tree hash: 2f7aec8fa3f0bf2cf1461d979da7783bdf2eace4 |
|
Could use a follow PR, especially after the linuxfoundation/lfevents#1107 |
|
Cc @dipesh-rawat for approval |
This PR is ready to address that issue. |
|
In a follow-up PR, please add this script to the scripts README.md https://github.com/kubernetes/website/blob/main/scripts/README.md |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: reylejano, singh1203 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Description
Replaces hardcoded KubeCon event links with an automated system that fetches and displays events from the Linux Foundation calendar.
Changes
content/en/_index.htmlkubecon-events.htmlshortcode for dynamic event renderingdata/kubecon_events.yamlcontaining structured event data generated from scripts/fetch_kubecon_eventsscripts/fetch_kubecon_events.pyto scrape and generate event dataTo Update Events
Preview
Related to #52376