Cloud-init bi-weekly status

Posted on Tue 29 October 2019 in status-meeting-minutes • 12 min read

Meeting information

Meeting summary

Previous Actions

The discussion about "Previous Actions" started at 16:26.

Recent Changes

The discussion about "Recent Changes" started at 16:36.

In-progress Development

The discussion about "In-progress Development" started at 16:41.

Community Charter

The discussion about "Community Charter" started at 16:49.

Upcoming meetings

The discussion about "Upcoming meetings" started at 16:57.

Upcoming Releases

The discussion about "Upcoming Releases" started at 16:58.

Office Hours (next ~30 mins)

The discussion about "Office Hours (next ~30 mins)" started at 17:00.

  • ACTION: cloud-init upstream discuss where/how/if to host documentation supported cloud-init versions for debian/suse/redhat/ubuntu/freebsd etc.
  • ACTION: cloud-init upstream discuss where/how/if to host documentation on supported and planned schedule for cloud-init versions for debian/suse/redhat/ubuntu/freebsd etc.
  • LINK: https://github.com/cloud-init/ubuntu-sru/blob/master/manual/ec2-sru-19.2.36.ubuntu2.txt
  • ACTION: mailing list update to cloud-init community about procedure/timeline for migration to github

Vote results

Done items

  • (none)

People present (lines said)

  • blackboxsw (130)
  • tribaal (34)
  • meetingology (8)
  • ubot5 (7)
  • Odd_Bloke (2)
  • chillysurfer (1)
  • rharper (0)

Full Log

16:20 <blackboxsw> #startmeeting Cloud-init bi-weekly status

16:20 <meetingology> Meeting started Tue Oct 29 16:20:09 2019 UTC. The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.

16:20 <meetingology>

16:20 <meetingology> Available commands: action commands idea info link nick

16:20 <blackboxsw> Boo! Welcome to the Halloween edition of cloud-init bi-weekly status meeting

16:20 <blackboxsw> #chair rharper

16:20 <meetingology> Current chairs: blackboxsw rharper

16:20 <blackboxsw> #chair Odd_Bloke

16:20 <meetingology> Current chairs: Odd_Bloke blackboxsw rharper

16:21 <blackboxsw> cloud-init upstream uses this meeting as a platform for community updates, feature/bug discussions, and an opportunity to get some extra input on current development.

16:21 <blackboxsw> shamefully I think we wall missed last week's status. So let's celebrate today instead to catch up with where cloud-init development is headed

16:22 <blackboxsw> So that we don't forget, let's set the topic for next status meeting.

16:22 <blackboxsw> Odd_Bloke: rharper does 11/12 look good?

16:22 <blackboxsw> as in, no holidays etc?

16:25 <blackboxsw> ok topic updated. Any concerns we can change it and send out an email to the mailing list

16:26 <blackboxsw> Feel free to interject at any time. Our typical format is the following: Previous Actions, Recent Changes, In-progress Development, Community Charter, Upcoming Meetings, Office Hours (~30 mins).

16:26 <blackboxsw> #topic Previous Actions

16:26 <blackboxsw> I have three actions from last meeting

16:26 <blackboxsw> #link https://cloud-init.github.io/status-2019-10-08.html#status-2019-10-08

16:28 * blackboxsw blackboxsw verify cpc image builds are unpaused/unblocked for Azure (we were able to confirm the the CPC clound image builds for Azure were on time for the two cloud-init SRUs and worked that communication side-channel with Microsoft.

16:28 <blackboxsw> We also established a point of contact via email for continued cloud image status updates

16:29 * blackboxsw cloud-init upstream review https://code.launchpad.net/~ahosmanmsft/cloud-init/+git/cloud-init/+merge/372957 . This branch is getting a bit of review work. It's looking pretty good and I expect we'll land it this week or next (Azure continuous integration testing support) Thanks ahosmanMSFT

16:30 <chillysurfer> nice, good work ahosmanMSFT

16:30 * blackboxsw upstream cloud-init email about 19.3 release date . rharper just sent out a message to announce the 19.3 upstream release date. on 11/05. https://lists.launchpad.net/cloud-init/msg00229.html

16:30 <blackboxsw> #link https://lists.launchpad.net/cloud-init/msg00229.html

16:30 <blackboxsw> thx chillysurfer :)

16:31 <blackboxsw> anyone with vested interested in getting branches into the 19.3 upstream release please ping us in this channel or the mailinglist to get branches reviewed and landed by Tuesday of next week

16:31 <blackboxsw> As a note, we are planning a 19.4 release by end of year too, so ~6 weeks later we can get branches into 19.4 that miss the 19.3 release date

16:33 <blackboxsw> and for those that aren't aware cloud-init's upstream releases generally are not feature-based but time based. Our goal is 4 time-based releases per year having the versioning schema <YEAR>.<minor_release_number> where minor_release_number will be [1-4]

16:34 <blackboxsw> if there are significant features that are near completion at a given timed release we will try our best to get those features/fixes vetted and landed.

16:35 <blackboxsw> shortly after each upstream release, we will publish that cloud-init version to the Ubuntu devel release (codenamed named Focal Fossa) our copr repos and SRU into Xenial, Bionic and Eoan.

16:35 <blackboxsw> ok long-winded. But I think that was our actions from last cloud-init status meeting.

16:35 <blackboxsw> next topic

16:36 <blackboxsw> #topic Recent Changes

16:37 <blackboxsw> The following branches have landed in tip since last meeting: via git log --since 2019-10-08

16:37 <blackboxsw> - net/netplan: use ipv6-mtu key for specifying ipv6 mtu values

16:37 <blackboxsw> - Fix usages of yaml, and move yaml_dump to safeyaml.dumps. (LP: #1849640)

16:37 <blackboxsw> - exoscale: Increase url_max_wait to 120s. [Chris Glass]

16:37 <blackboxsw> - net/sysconfig: fix available check on SUSE distros

16:37 <blackboxsw> [Robert Schweikert] (LP: #1849378)

16:37 <blackboxsw> - docs: Fix incorrect Azure IMDS IP address [Joshua Powers] (LP: #1849508)

16:37 <blackboxsw> - introduce .travis.yml

16:37 <ubot5> Launchpad bug 1849640 in cloud-init "security scan reported insecure yaml load method usage in latest cloud-init code" [Undecided,Fix committed] https://launchpad.net/bugs/1849640

16:37 <blackboxsw> - net: enable infiniband support in eni and sysconfig renderers

16:37 <blackboxsw> [Darren Birkett] (LP: #1847114)

16:37 <ubot5> Launchpad bug 1849378 in cloud-init "sysconfig renderer not detected on SLE distros" [Medium,Fix committed] https://launchpad.net/bugs/1849378

16:37 <blackboxsw> - guestcust_util: handle special characters in config file [Xiaofeng Wang]

16:37 <blackboxsw> - fix some more typos in comments [Dominic Schlegel]

16:37 <ubot5> Launchpad bug 1849508 in cloud-init "Azure IMDS documentation wrong about IP address" [High,Fix committed] https://launchpad.net/bugs/1849508

16:37 <blackboxsw> - replace any deprecated log.warn with log.warning

16:37 <blackboxsw> [Dominic Schlegel] (LP: #1508442)

16:37 <blackboxsw> - net: handle openstack dhcpv6-stateless configuration

16:37 <blackboxsw> [Harald Jensås] (LP: #1847517)

16:37 <ubot5> Launchpad bug 1847114 in cloud-init "Infiniband network devices not configured correctly on Ubuntu" [Medium,Fix committed] https://launchpad.net/bugs/1847114

16:37 <blackboxsw> - Add .venv/ to .gitignore [Dominic Schlegel]

16:37 <blackboxsw> - Small typo fixes in code comments. [Dominic Schlegel]

16:37 <blackboxsw> - cloud_test/lxd: Retry container delete a few times

16:37 <blackboxsw> - Add Support for e24cloud to Ec2 datasource. (LP: #1696476)

16:37 <ubot5> Launchpad bug 1508442 in python-watcherclient "LOG.warn is deprecated" [Low,In progress] https://launchpad.net/bugs/1508442

16:37 <ubot5> Launchpad bug 1847517 in cloud-init "cloudinit/net/sysconfig.py write incorrect config for dhcp-stateless openstack subnets" [Medium,Fix committed] https://launchpad.net/bugs/1847517

16:37 <ubot5> Launchpad bug 1696476 in cloud-init "Identification of e24cloud platform as using Ec2 datasource" [Low,Fix committed] https://launchpad.net/bugs/1696476

16:39 <blackboxsw> some good incremental improvements on SuSE, FreeBSD,Ubuntu Azure, Ec2, OpenStack and Exoscale

16:40 <blackboxsw> Thanks Dominic, Robert, XiaoFeng, Harald and Darren and Scott for the contibutions this round!

16:41 <blackboxsw> #topic In-progress Development

16:41 <blackboxsw> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin

16:42 <blackboxsw> We keep track of most of cloud-init's development efforts on the cloud-init and curtin trello board above.

16:43 <blackboxsw> And the active review queue for cloud-init is a good representation of what should land shortly.

16:43 <blackboxsw> #link http://bit.ly/ci-reviews

16:44 <blackboxsw> The following work-streams are currently in progress:

16:44 <blackboxsw> - CI for Azure (thanks ahosmanMSFT)

16:44 <blackboxsw> - Openstack networking v2 support (which allows openstack to talk v2 network config 'netplan' )

16:45 <blackboxsw> - Azure async disk mounts

16:45 <blackboxsw> - cloud-init boot performance improvements

16:46 <blackboxsw> As features related to those streams make progress we'll be updating the trello board and dragging cards to the done lane

16:47 <blackboxsw> There are also a number of branches for FreeBSD support up that we need to shepherd and merge I think

16:48 <blackboxsw> #link https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368507

16:48 <blackboxsw> #link https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641

16:48 <blackboxsw> We'll be able to review some of the outstanding branches during Office hours today

16:49 <blackboxsw> #topic Community Charter

16:50 <blackboxsw> This point in the meeting is essentially an open reminder of the ongoing community charter work and low hanging fruit that everyone is invited to participate in.

16:51 <blackboxsw> We has a list of tasks/features and bugs that are considered low-hanging-fruit and highly decoupled work items that anyone is welcome to grab. They are listed in the "Community low hanging fruit" column on our trello board

16:51 <blackboxsw> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin

16:52 <blackboxsw> Our current community goal is to scrub, update and correct all datasource docs seen @ https://cloudinit.readthedocs.io/en/latest/topics/datasources.html

16:52 <tribaal> That's nice!

16:52 <blackboxsw> all help is appreciated especially from the datasource owners

16:52 <blackboxsw> \o tribaal.

16:52 <tribaal> o/

16:53 <tribaal> sorry, I'm following along, didn't mean to interrupt

16:53 <blackboxsw> additionally there is a really long tail of work needed to update each cloud-init config module (cloudinit/config/cc_*py) with a json schema definition.

16:55 <blackboxsw> There are around 50 modules that would need a jsonschema definition so that we can promote our cloud-init devel schema command to it can properly vet any #cloud-config a user provides to annotate user errors

16:55 <blackboxsw> any help grabbing individualy config modules and adding that would really add value to cloud-init to help users solve their own errors as invalid #cloud-config is one of our highest frequency issues

16:56 <blackboxsw> example https://trello.com/c/NENad4m3/1172-schema-add-initial-schema-definition-for-ccpowerstatechange

16:56 <blackboxsw> no worries, interruptions welcome

16:56 <blackboxsw> think that about wraps up community charter.

16:56 <blackboxsw> two to topics to go

16:57 <blackboxsw> #topic Upcoming meetings

16:57 <blackboxsw> I already updated the topic. 2 weeks from today 11/05 10:15 utc cloud-init status. We'll be here; feel free to drop in

16:58 <blackboxsw> #topic Upcoming Releases

16:59 <blackboxsw> Again cloud-init upstream 19.3 scheduled to be cut next Tuesday 11/05 at EOD. Raise awareness in #cloud-init IRC channel or on the mailing list for bugs and feature branches that need attention prior to cutting the release

16:59 <blackboxsw> Immediately after release, we'll publish to Ubuntu's devel release Focal Fossa and our copr repo

16:59 <tribaal> nice! this answers one of the questions I had coming in today :)

16:59 <blackboxsw> #link https://copr.fedorainfracloud.org/coprs/g/cloud-init/el-testing/

16:59 <blackboxsw> good deal tribaal

17:00 <blackboxsw> and again, upstream will schedulae 19.4 release probably ~6 weeks after 19.3 so not long to wait if folks miss 19.3

17:00 <blackboxsw> schedule even

17:00 <blackboxsw> #topic Office Hours (next ~30 mins)

17:01 <blackboxsw> bring your bugs, discussions, features, concerns here about cloud-init life the universe and everything. upstream will spend this time actively grooming the review queue to see if we can't unblock developers

17:02 <tribaal> blackboxsw: I suspect the plan for y'all is to SRU 19.3 into the various Ubuntues after it lands in focal?

17:03 <tribaal> my latest bugfix, although small, impacts one of our customers so I'd like to give them a heads up in that direction

17:03 <blackboxsw> tribaal: yep sorry, I mentioned earlier, but should have mentioned in the last topic

17:03 <tribaal> no worries

17:03 <tribaal> it's the logical follow-up to a release - so it was surely implied

17:04 <blackboxsw> SRU will be scheduled shortly after 19.3 upstream cut I presume. We prefer to keep time between SRUs smaller to make the verification more manageable

17:04 <tribaal> blackboxsw: do you know how/when cloud-init lands in other distros, as a matter of curiosity?

17:04 <blackboxsw> I think now that Ubuntu Eoan is released and post-release dust is settling we'll have bandwidth to cover that

17:05 <blackboxsw> tribaal: robjo knows for SLES. otubo (hrm not here today) is managing RHEL/CentOS release schedules

17:06 <blackboxsw> and debian is trying to shore up to 19.X I believe.

17:06 <tribaal> ack

17:06 <blackboxsw> but I think other distros are delayed (for good reasons: testing/validation/stability/etc).

17:07 <tribaal> sure, that would make sense

17:07 <tribaal> more QA is good :)

17:07 * blackboxsw finds myself wanting a current support matrix on cloudinit.readthedocs.io

17:07 <blackboxsw> or somewhere

17:07 <tribaal> that would be incredibly useful to us :)

17:08 <blackboxsw> agreed. Maybe we can take an action to at least discuss what that would look like (and where to host it)

17:08 <tribaal> tracking where and when our new datasource is available is a bit of a mystery right now :)

17:08 <tribaal> (except for Ubuntus)

17:09 <blackboxsw> #action cloud-init upstream discuss where/how/if to host documentation supported cloud-init versions for debian/suse/redhat/ubuntu/freebsd etc.

17:09 * meetingology cloud-init upstream discuss where/how/if to host documentation supported cloud-init versions for debian/suse/redhat/ubuntu/freebsd etc.

17:09 <tribaal> blackboxsw: another question since I have the mic: would it be useful to you to add us to your CI pipeline?

17:09 <blackboxsw> we can come back with some kindof answer for that question (as I end up doing more work like lxc launch distroX) frequently to try to see

17:11 <tribaal> blackboxsw: cool - but that's more a measure rather than an expected date right? so it wouldn't help our marketing people to plan blog posts for instance (which is why I'm asking this in the first place - we have some comms ready to go around a feature that requires the new datasource)

17:11 <blackboxsw> tribaal: I think it would be useful to extend our cloudinit/tests/cloud_tests to allow us to manually kick off exoscale runs during SRU validation. Especially if you are finding that our upstream changes are breaking your platform

17:12 <blackboxsw> tribaal: yeah let me restate the action then

17:12 <blackboxsw> #action cloud-init upstream discuss where/how/if to host documentation on supported and planned schedule for cloud-init versions for debian/suse/redhat/ubuntu/freebsd etc.

17:12 * meetingology cloud-init upstream discuss where/how/if to host documentation on supported and planned schedule for cloud-init versions for debian/suse/redhat/ubuntu/freebsd etc.

17:12 <blackboxsw> per scheduling we at least could start filling in some gaps there on cloud-init planned distro releases

17:13 <tribaal> blackboxsw: so, you're basically never breaking our platform - clearly :) But being part of the SRU validation would be good maybe. I don't know how much work that is however (I don't know what I'm asking).

17:13 <tribaal> is that something we can do ourselves?

17:13 <tribaal> (code wise at least)

17:14 <blackboxsw> tribaal: codewise, if you are talking about manual tests, we generally run something like the following:

17:14 <blackboxsw> #link https://github.com/cloud-init/ubuntu-sru/blob/master/manual/ec2-sru-19.2.36.ubuntu2.txt

17:15 <blackboxsw> if you wanted to write up an example manual sru test and propose it to https://github.com/cloud-init/ubuntu-sru/

17:15 <blackboxsw> you could follow the examples in https://github.com/cloud-init/ubuntu-sru/blob/master/manual/ec2-sru-19.2.36.ubuntu2.txt

17:15 <blackboxsw> oops wrong paste

17:15 <blackboxsw> you could follow the examples in : https://github.com/cloud-init/ubuntu-sru/tree/master/sru-templates/manual

17:16 <tribaal> oh, I see

17:16 <tribaal> so thankfully it's only manual in the "manually run a script" sense :)

17:16 <blackboxsw> and create one for exoscale. that's lowest hanging fruit I think. that'd increase chances of including a manual test in the SRU process. but, we might have to ping you for a recommendation on who could run that on our behalf

17:17 <tribaal> yes, that would work (myself, and a handful of colleagues would be happy to I guess)

17:17 <blackboxsw> that's lowest hanging fruit. next step up would be to add a platform definition that can automatically drive Exoscale like ahosmanMSFT has for Azure in progress at https://code.launchpad.net/~ahosmanmsft/cloud-init/+git/cloud-init/+merge/372957

17:18 <blackboxsw> having the cloud_tests platform definition would allow us our you to more easily extend a jenkins CI job to drive the cloud tests

17:19 <tribaal> ah nice, that makes sense

17:19 <blackboxsw> tribaal: the manual SRU verification script writing really helps (but is not mandatory) prior to extending tests/cloud_tests. having a manual script helps codify what we really might need to do to fully support a platform in tests/cloud_tests.

17:20 <tribaal> yeah that sounds like a good first step, and we can expand later on if/when needed

17:24 <blackboxsw> +1 tribaal

17:24 <blackboxsw> btw good deal that you were able to run into Odd_Bloke on your travels. Always good to see a familiar face

17:24 <tribaal> yeah that was a crazy coincidence

17:25 <tribaal> my brother got married in Odd_Bloke 's backyard :)

17:25 <tribaal> I wish we could do cross-company colocated sprints :P

17:26 <blackboxsw> heh, I think we can call those #cloud-init-summits :)

17:26 <tribaal> ohhh yeah :)

17:26 <tribaal> when's the next one?

17:26 <blackboxsw> we'll have to get invites together for the next one (Fall 2020)

17:27 <tribaal> noted :)

17:29 <blackboxsw> Ohh right and BIG NEWS that I forgot to highlight: cloud-init is moving from launchpad to github. We discussed this at the last cloud-init summit a couple of weeks ago.

17:29 <blackboxsw> Odd_Bloke: and rharper have been working on getting initial travis.yml together for allowing travis to run our CI tests.

17:30 <tribaal> ohhh big news indeed!

17:30 <Odd_Bloke> That has, indeed, landed. You can see it at https://github.com/cloud-init/cloud-init/blob/master/.travis.yml (it's also in the Launchpad git repo, we're still syncing from there to GH ATM :).

17:31 <Odd_Bloke> And you can see that we're green at https://travis-ci.org/cloud-init/cloud-init

17:31 <blackboxsw> We are currently sorting out the rest of our CI, but the expectation for the community is that we'll soon have an announcement and simple procedure to get folks contributing with pull requests @ https://github.com/cloud-init/cloud-init instead of posting branch review requests at https://git.launchpad.net/cloud-init

17:31 <tribaal> very nice!

17:33 <blackboxsw> There will likely be a small transition set of steps needed for existing launchpad users to authenticate their github account because of the Canonical's contributor license agreement (CLA). But we are sorting that imminently

17:33 <tribaal> ah that would make sese yeah

17:34 <blackboxsw> we'll make every effort to reduce the amount of work for in-flight branches and get landed what we can prior to that transition to github. So, expect before next cloud-init status meeting that we have details about the switch

17:35 <blackboxsw> #action mailing list update to cloud-init community about procedure/timeline for migration to github

17:35 * meetingology mailing list update to cloud-init community about procedure/timeline for migration to github

17:36 <blackboxsw> we really want to leverage the active github community, public CI results, and ease of use and github is the best path forward for that

17:36 <blackboxsw> thanks Odd_Bloke and rharper for all the work there so far

17:38 <blackboxsw> well. I think that about wraps the status meeting for this episode. Hope Halloween has more treat than trick for folks. See you next 11/05 16:15 UTC.

17:38 <blackboxsw> thanks for participating chillysurfer & tribaal

17:38 <blackboxsw> #endmeeting

Generated by MeetBot 0.1.5 (http://wiki.ubuntu.com/meetingology)