Cloud-init bi-weekly status

Posted on Mon 08 July 2019 in status-meeting-minutes • 10 min read

Meeting information

Meeting summary

Previous Actions

The discussion about "Previous Actions" started at 16:18.

Recent Changes

The discussion about "Recent Changes" started at 16:22.

In Progress Development

The discussion about "In Progress Development" started at 16:27.

Office Hours (next ~30 mins)

The discussion about "Office Hours (next ~30 mins)" started at 16:48.

Vote results

Action items, by person

Done items

  • (none)

People present (lines said)

  • blackboxsw (89)
  • Odd_Bloke (13)
  • bitfehler (10)
  • tribaal (8)
  • AnhVoMSFT (7)
  • chad-aws (6)
  • meetingology (5)
  • ubot5 (3)
  • robjo (2)
  • rharper (2)

Full Log

16:16 <blackboxsw> #startmeeting Cloud-init bi-weekly status

16:16 <meetingology> Meeting started Mon Jul 8 16:16:57 2019 UTC. The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.

16:16 <meetingology>

16:16 <meetingology> Available commands: action commands idea info link nick

16:17 <Odd_Bloke> bitfehler: It works for me on an Ubuntu eoan system, locally.

16:17 <blackboxsw> hi folks, welcome to another cloud-init community status meeting. All discussions and interjections welcome.

16:17 <Odd_Bloke> bitfehler: What version of Python 3 are you using?

16:17 <blackboxsw> loud-init upstream uses this meeting as a platform for community updates, feature/bug discussions, and an opportunity to get some extra input on current development.

16:17 <blackboxsw> our format is the following topics: Previous Actions, Recent Changes, In-progress Development, Office Hours

16:17 <blackboxsw> anyone is welcome to participate, interject, make suggestions or ask questions

16:18 <blackboxsw> we host the meeting every two weeks at the date and time indicated in the IRC channel topic ^

16:18 <blackboxsw> #topic Previous Actions

16:19 <blackboxsw> I'm looking through our meeting minutes now from the previous meeting

16:19 <blackboxsw> #link https://cloud-init.github.io/status-2019-06-24.html#status-2019-06-24

16:19 <blackboxsw> Touch base with AnhVoMSFT by next status on priority of https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1832381

16:19 <ubot5> Ubuntu bug 1832381 in cloud-init (Ubuntu) "vm fails to boot due to conflicting network configuration when user switches from netplan to eni" [Undecided,Incomplete]

16:20 <blackboxsw> I think AnhVoMSFT may have been looking to get cloud-init logs on a system reproducing this problem

16:20 <blackboxsw> so let's carry this over for next meeting and this bug is marked incomplete until we have some cloud-init logs to debug

16:21 <blackboxsw> #action query on incomplete https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1832381 with AnhVoMSFT to see if this needs priority

16:21 * meetingology query on incomplete https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1832381 with AnhVoMSFT to see if this needs priority

16:21 <ubot5> Ubuntu bug 1832381 in cloud-init (Ubuntu) "vm fails to boot due to conflicting network configuration when user switches from netplan to eni" [Undecided,Incomplete]

16:21 <blackboxsw> no further actions listed from last meeting

16:22 <blackboxsw> #topic Recent Changes

16:22 <AnhVoMSFT> blackboxsw I did hear back from networking on how to trigger mac address change, so I'll work on that this week and get the logs attached

16:22 <blackboxsw> excellent AnhVoMSFT thanks!

16:22 <blackboxsw> and thanks for joining the meeting

16:22 <AnhVoMSFT> the harder it is to reproduce, the better, since it's unlikely to affect that many customers

16:22 <blackboxsw> +1

16:23 <blackboxsw> the following are commits that landed in tip of master for cloud-init upstream

16:23 <blackboxsw> - Add missing dsname for Hetzner Cloud datasource [Markus Schade]

16:24 <blackboxsw> - doc: indicate that netplan is default in Ubuntu now [Daniel Watkins]

16:24 <blackboxsw> - azure: add region and AZ properties from imds compute location metadata

16:24 <blackboxsw> [Chad Smith]

16:24 <blackboxsw> - sysconfig: support more bonding options [Penghui Liao]

16:24 <blackboxsw> - cloud-init-generator: use libexec path to ds-identify on redhat systems

16:24 <blackboxsw> [Ryan Harper] (LP: #1833264)

16:24 <blackboxsw> - tools/build-on-freebsd: update to python3 [Gonéri Le Bouder]

16:24 <ubot5> Ubuntu bug 1833264 in cloud-init "cloud-init-generator hardcodes path to ds-identify" [Undecided,Fix committed]

16:25 <blackboxsw> though I think the bottom two of those commits I may have reported last meeting

16:26 <blackboxsw> beyond that I know that paride has resolved a couple of issues with our CI infrastructure not cleaning up stale containers which would have triggered a number of CI failures over the last few weeks.

16:27 <blackboxsw> I think that about wraps 'completed' work in tip.

16:27 <blackboxsw> #topic In Progress Development

16:28 <blackboxsw> We mentioned this last meeting, there are a couple of longer features we are working on that will hit cloud-init tip soon.

16:28 <blackboxsw> we track our work on trello at the following url

16:28 <blackboxsw> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin

16:29 <blackboxsw> In trying to enable secondary IP configuration on Azure platforms, we discovered a systemd-networkd bug related to classless routes not getting the appropriate source IP configuration

16:29 <blackboxsw> per this card

16:29 <blackboxsw> #link https://trello.com/c/RhevWnHx/1064-azure-imds-handle-multiple-default-routes-static-ips-in-primary-subnet

16:30 <blackboxsw> Dan Streetman filed a bug and upstream systemd fix for this https://github.com/systemd/systemd/issues/12969

16:31 <blackboxsw> and he's working on getting that released into Ubuntu Eoan. cloud-init may need a minor fix to only render static IPs if systemd-networkd version contains the latest fix.

16:31 <chad-aws> secondary addresses on azure too, or aws?

16:31 <blackboxsw> chad-aws: I also have just pushed a branch for review that will add secondary IPs from AWS's Datasource as well

16:31 <blackboxsw> since we had context on the netplan we need to generate that

16:31 <chad-aws> ok

16:32 <blackboxsw> #link https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/369792

16:33 <blackboxsw> chad-aws: the one question I think we might have to discuss related to the above branch is whether or not cloud-init on older LTSes (Xenial, Bionic) should change behavior to attempt rendering secondary IP information or not (because that would be a change in behavior)

16:34 <blackboxsw> generally we try to retain existing behavior on old Ubuntu LTS releases so we don't surprise folks who may have worked around previous limitations of cloud-init (like manually/scripted adding their own secondary IP information)

16:35 <chad-aws> (Note I am not the chad.smith above, but I am interested too.)

16:35 <blackboxsw> hehe, yes, I am not talking to myself (blackboxsw == chad.smith)

16:36 <blackboxsw> also in progress is rharper's good work on teasing out async mount functionality per the following

16:36 <blackboxsw> #link https://trello.com/c/TMK5ZDMf/1108-azure-async-disk-mounts

16:36 <chad-aws> I think different config files should make different behavior and that's okay.

16:36 <blackboxsw> +1 chad-aws

16:37 <blackboxsw> the async mount feature is the ability to allow cloud-init defer some disk mounts until later in the cloud-init stages to allow folks to ssh to the vms earlier in the boot process

16:38 <blackboxsw> as some systems with lots of mounts may block for a while trying to mount and format disks

16:39 <blackboxsw> while this approach is tagged as azure, rharper is approaching it in a generic way that should make this functionality accessible to many platforms

16:40 <blackboxsw> paride: rharper Odd_Bloke anything else I'm forgetting in progress?

16:40 <rharper> blackboxsw: right; it's changes to cc_disk_setup/cc_mount handling

16:40 <rharper> blackboxsw: nothing I can think of

16:41 <blackboxsw> we probably should cut an Ubutuu Eoan upload of cloud-init tip soon, but I don't know when that should be scheduled?

16:41 <Odd_Bloke> I'm doing some work to modify the way we determine network config sources, but that shouldn't affect any data sources that don't opt in to it.

16:41 <AnhVoMSFT> i looked at it briefly, this involves calling a systemd unit to format/mount - do we report error back to cloud-init ?

16:41 * tribaal raises hand

16:41 <tribaal> is there any rough estimation on where that tip cut would be?

16:41 <blackboxsw> tribaal: ahh yes awesome, forgot. tribaal has a new datasource

16:41 <tribaal> yes, thanks a lot for your reviews everyone

16:42 <tribaal> nice working with you guys again :)

16:42 <blackboxsw> #link https://code.launchpad.net/~tribaal/cloud-init/+git/cloud-init/+merge/369516

16:42 <blackboxsw> for Exoscale ^

16:42 <tribaal> Ideally we'd target the next release for our datasource as well, but of course it's not landed yet so if the cut is e.g. tomorrow it might be a bit tight :)

16:42 <Odd_Bloke> Specifically, if a platform does any networking setup during the initramfs (such as for iSCSI), cloud-init will only consider the initramfs-provided configuration at the moment.

16:42 <blackboxsw> tribaal: for eoan, we can just grab tip of master at anypoint for an upload since it

16:43 <blackboxsw> is still a development release

16:43 <tribaal> ack

16:43 <blackboxsw> so it's super low weight for our release (and we should do it often)

16:43 <blackboxsw> also we have planned/upcoming an SRU into Xenial/Bionic/Disco on July 15th (so next week)

16:44 <tribaal> as far as SRUs are concerned, would that typically be something that would be backported, or not?

16:44 <Odd_Bloke> I'm making it possible for data sources to specify a different order for the network config sources, so that platforms where iSCSI is the default can have their data source's network config preferred.

16:44 <blackboxsw> tribaal: and others: cloud-init SRUs our tip into every release due to an SRU exception that we have with Ubuntu

16:44 <blackboxsw> so we upload latest code into each SRU target series for ubuntu

16:45 <blackboxsw> though we may patch/disable some functionality that is deemed a change in behavior from previous release

16:45 <tribaal> blackboxsw: understood, thanks. I'll schedule some time for myself to work on it "full time" as much as possible, so we can make the cut.

16:45 <Odd_Bloke> I'm also splitting apart explicitly-from-the-user cmdline configuration (i.e. network-data=...) from the initramfs-provided values (ip=... or iBFT), so that data sources can be configured to still allow explicit user network configuration to override data source network configuration.

16:45 <blackboxsw> here's our SRU process for those interested

16:45 <blackboxsw> #link https://wiki.ubuntu.com/CloudinitUpdates

16:46 <blackboxsw> so this meeting (and an email to cloud-init mailinglist) will serve as a call for branches/features for SRU.

16:46 <Odd_Bloke> But as I said previously, this is all just refactoring except for the specific places we need this functionality.

16:46 <AnhVoMSFT> I sent a merge proposal for adding some more telemetry for Azure: https://code.launchpad.net/~vtqanh/cloud-init/+git/cloud-init/+merge/369785 - would appreciate a review. Trying to make it in before next SRU

16:46 <AnhVoMSFT> also will send a merge proposal today or tomorrow on the case sensitivity issue when checking is_new_instance

16:46 <blackboxsw> if folks are interested in getting a specific feature/bug or branch reviewed/fixed and published, please raise your hand, ping in channel or send email to the mailing list to make sure the need is not forgotten

16:47 <blackboxsw> #action review Azure telemetry branch https://code.launchpad.net/~vtqanh/cloud-init/+git/cloud-init/+merge/369785

16:47 * meetingology review Azure telemetry branch https://code.launchpad.net/~vtqanh/cloud-init/+git/cloud-init/+merge/369785

16:47 <bitfehler> i feel a bit awkward stepping in here, but i kind of do

16:47 <blackboxsw> AnhVoMSFT: we also have reviewed samgilson's branch on a new cloud-init analyze subcommand for boot performance

16:48 <bitfehler> sorry, i am pretty new to cloud-init, but i did open a merge proposal today

16:48 <blackboxsw> bitfehler: no worries, conversations gotta happen :)

16:48 <Odd_Bloke> #link https://code.launchpad.net/~bitfehler/cloud-init/+git/cloud-init/+merge/369814

16:48 <Odd_Bloke> (Can I do that, or only blackboxsw?)

16:48 <AnhVoMSFT> thanks blackboxsw, last I synced with him, Sam is actively working on addressing your comments

16:48 <blackboxsw> I'll tick the topic to office hours which is really just open season for discussion or reviews

16:48 <robjo> Does anyone know if VMware has any plans to move their source from https://github.com/vmware/cloud-init-vmware-guestinfo to be included in upstream cloud-init?

16:48 <blackboxsw> #topic Office Hours (next ~30 mins)

16:51 <bitfehler> i think my main question would have been how to best get in touch with you, but i think i found the answer already :)

16:51 <blackboxsw> hrm, first I've seen of that robjo . maybe we can ping Sankar on that to see what the motivation is there

16:51 <blackboxsw> bitfehler: either here or mail cloud-init@lists.launchpad.net

16:52 <robjo> I was just made aware of this last week via "please create a packge' to which my initial answer is No, the VMware code should be in cloud-init proper

16:53 <blackboxsw> per vmware,yeah that makes sense for them to try to get that cloud-init datasource upstream. forcing every distro to do their packaging for them is not really the right solution

16:54 <bitfehler> one other thing i was wondering: do you feel a plain systemd-networkd network renderer would make sense? i wrote a very basic one trying to get better cloud-init support for Arch Linux

16:54 <bitfehler> it sort of works, but I am not sure whether this goal worth pursuing?

16:57 <blackboxsw> bitfehler: interesting, right so we have netplan render which ultimately renders networkd on our behalf on ubuntu,

16:58 <blackboxsw> if direct networkd render is the only way to support network config on Arch linux I don't see why we wouldn't want that.... rharper or Odd_Bloke? (maybe I'm missing the concern)

16:59 <bitfehler> i saw the comments about networkd above, but i am not familiar with netplan. it is another layer in between, right?

17:00 <Odd_Bloke> bitfehler: netplan takes v2 network configuration and renders it for a target backend. The two supported backends ATM are networkd and NetworkManager.

17:00 <blackboxsw> https://netplan.io for more info

17:00 <bitfehler> oh, wow, i overlooked that. so it doesn't need any additional software?

17:01 <AnhVoMSFT> it does need netplan

17:01 <bitfehler> oh wait, netplan is a renderer itself, and that supports the two backends

17:02 <Odd_Bloke> Right, netplan is the intended way for cloud-init to render network config for networkd.

17:03 <bitfehler> ok, got it. i guess i could also look into porting netplan to arch then? not sure what an effort that would be

17:05 <Odd_Bloke> IMO, that would be the best way to go, if it's tractable.

17:05 <Odd_Bloke> We ideally wouldn't reimplement netplan's networkd renderer in cloud-init. :)

17:05 <bitfehler> i'll let you know soon ;)

17:08 <chad-aws> I guess this should go both ways. Is there anything AWS EC2 can do better or different?

17:25 <blackboxsw> nice chad-aws, sorry I got pulled into a second meeting. chad-aws I did have a question about metadata versioning in aws.

17:26 <blackboxsw> how are new features communicated for a new metadata version

17:26 <blackboxsw> I reference https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

17:26 <blackboxsw> but it didn't have a specific note on the changes added for 2018-09-24 which I used in my branch

17:27 <blackboxsw> only the first version that a field was introduced, not that local_ipv4s changed value from string to list in 2018-09-24.

17:27 <blackboxsw> so I guess my question is, is there a better source of truth for ec2 metadata values that we should be looking at

17:28 <blackboxsw> chad-aws: sorry (I realize that's a question out of left field)

17:31 <blackboxsw> I think I'll wrap the meeting here. Thank you all again for the attending this week.

17:31 <chad-aws> thx

17:31 <blackboxsw> Again plan for cloud-init 19.2 SRU is scheduled for next week so any features/branches that need landing or review should be raised here or on the mailing list

17:32 <blackboxsw> minutes will be posted at https://cloud-init.github.io

17:32 <blackboxsw> #endmeeting

Generated by MeetBot 0.1.5 (http://wiki.ubuntu.com/meetingology)