Planet Ensigogne

10 September 2014

Lucas Nussbaum

Will the packages you rely on be part of Debian Jessie?

The start of the jessie freeze is quickly approaching, so now is a good time to ensure that packages you rely on will the part of the upcoming release. Thanks to automated removals, the number of release critical bugs has been kept low, but this was achieved by removing many packages from jessie: 841 packages from unstable are not part of jessie, and won’t be part of the release if things don’t change.

It is actually simple to check if you have packages installed locally that are part of those 841 packages:

  1. apt-get install how-can-i-help (available in backports if you don’t use testing or unstable)
  2. how-can-i-help --old
  3. Look at packages listed under Packages removed from Debian ‘testing’ and Packages going to be removed from Debian ‘testing’

Then, please fix all the bugs :-) Seriously, not all RC bugs are hard to fix. A good starting point to understand why a package is not part of jessie is tracker.d.o.

On my laptop, the two packages that are not part of jessie are the geeqie image viewer (which looks likely to be fixed in time), and josm, the OpenStreetMap editor, due to three RC bugs. It seems much harder to fix… If you fix it in time for jessie, I’ll offer you a $drink!

10 September 2014 à 19:28

01 September 2014

Lucas Nussbaum

Debian releases used by popcon participants, V2

In March 2013 I looked at Debian releases used by popcon participants. I’ve just re-done the same analysis. Please see the previous post on this topic for details.

releases

01 September 2014 à 01:53

31 August 2014

Lucas Nussbaum

Debian trivia

After an intensive evening of brainstorming by the 5th floor cabal, I am happy to release the very first version of the Debian Trivia, modeled after the famous TCP/IP Drinking Game. Only the questions are listed here — maybe they should go (with the answers) into a package? Anyone willing to co-maintain? Any suggestions for additional questions?

31 August 2014 à 08:42

24 August 2014

Lucas Nussbaum

on the Dark Ages of Free Software: a “Free Service Definition”?

Stefano Zacchiroli opened DebConf’14 with an insightful talk titled Debian in the Dark Ages of Free Software (slides available, video available soon).

He makes the point (quoting slide 16) that the Free Software community is winning a war that is becoming increasingly pointless: yes, users have 100% Free Software thin client at their fingertips [or are really a few steps from there]. But all their relevant computations happen elsewhere, on remote systems they do not control, in the Cloud.

That give-up on control of computing is a huge and important problem, and probably the largest challenge for everybody caring about freedom, free speech, or privacy today. Stefano rightfully points out that we must do something about it. The big question is: how can we, as a community, address it?

Towards a Free Service Definition?

I believe that we all feel a bit lost with this issue because we are trying to attack it with our current tools & weapons. However, they are largely irrelevant here: the Free Software Definition is about software, and software is even to be understood strictly in it, as software programs. Applying it to services, or to computing in general, doesn’t lead anywhere. In order to increase the general awareness about this issue, we should define more precisely what levels of control can be provided, to understand what services are not providing to users, and to make an informed decision about waiving a particular level of control when choosing to use a particular service.

Benjamin Mako Hill pointed out yesterday during the post-talk chat that services are not black or white: there aren’t impure and pure services. Instead, there’s a graduation of possible levels of control for the computing we do. The Free Software Definition lists four freedoms — how many freedoms, or types of control, should there be in a Free Service Definition, or a Controlled-Computing Definition? Again, this is not only about software: the platform on which a particular piece of software is executed has a huge impact on the available level of control: running your own instance of WordPress, or using an instance on wordpress.com, provides very different control (even if as Asheesh Laroia pointed out yesterday, WordPress does a pretty good job at providing export and import features to limit data lock-in).

The creation of such a definition is an iterative process. I actually just realized today that (according to Wikipedia) the very first occurrence of an attempt at a Free Software Definition was published in 1986 (GNU’s bulletin Vol 1 No.1, page 8) — I thought it happened a couple of years earlier. Are there existing attempts at defining such freedoms or levels of controls, and at benchmarking such criteria against existing services? Such criteria would not only include control over software modifications and (re)distribution, but also likely include mentions of interoperability and open standards, both to enable the user to move to a compatible service, and to avoid forcing the user to use a particular implementation of a service. A better understanding of network effects is also needed: how much and what type of service lock-in is acceptable on social networks in exchange of functionality?

I think that we should inspire from what was achieved during the last 30 years on Free Software. The tools that were produced are probably irrelevant to address this issue, but there’s a lot to learn from the way they were designed. I really look forward to the day when we will have:

Exciting times!

24 August 2014 à 15:39

15 August 2014

Olivier Crête

GNOME.Asia Summit 2014

Everyone has been blogging about GUADEC, but I’d like to talk about my other favorite conference of the year, which is GNOME.Asia. This year, it was in Beijing, a mightily interesting place. Giant megapolis, with grandiose architecture, but at the same time, surprisingly easy to navigate with its efficient metro system and affordable taxis. But the air quality is as bad as they say, at least during the incredibly hot summer days where we visited.

The conference itself was great, this year, co-hosted with FUDCon’s asian edition, it was interesting to see a crowd that’s really different from those who attend GUADEC. Many more people involved in evangelising, deploying and using GNOME as opposed to just developing it, so it allows me to get a different perspective.

On a related note, I was happy to see a healthy delegation from Asia at GUADEC this year!

Sponsored by the GNOME Foundation

15 August 2014 à 04:50

17 July 2014

Vincent Untz

Stepping down as openSUSE Board Chairman

Two years ago, I got appointed as chairman of the openSUSE Board. I was very excited about this opportunity, especially as it allowed me to keep contributing to openSUSE, after having moved to work on the cloud a few months before. I remember how I wanted to find new ways to participate in the project, and this was just a fantastic match for this. I had been on the GNOME Foundation board for a long time, so I knew it would not be easy and always fun, but I also knew I would pretty much enjoy it. And I did.

Fast-forward to today: I'm still deeply caring about the project and I'm still excited about what we do in the openSUSE board. However, some happy event to come in a couple of months means that I'll have much less time to dedicate to openSUSE (and other projects). Therefore I decided a couple of months ago that I would step down before the end of the summer, after we'd have prepared the plan for the transition. Not an easy decision, but the right one, I feel.

And here we are now, with the official news out: I'm no longer the chairman :-) (See also this thread) Of course I'll still stay around and contribute to openSUSE, no worry about that! But as mentioned above, I'll have less time for that as offline life will be more "busy".

openSUSE Board Chairman at oSC14

openSUSE Board Chairman at oSC14

Since I mentioned that we were working on a transition... First, knowing the current board, I have no doubt everything will be kept pushed in the right direction. But on top of that, my good friend Richard Brown has been appointed as the new chairman. Richard knows the project pretty well and he has been on the board for some time now, so is aware of everything that's going on. I've been able to watch his passion for the project, and that's why I'm 100% confident that he will rock!

17 July 2014 à 14:40

09 April 2014

Lucas Nussbaum

speedtest.net, or how not to do bandwidth tests

While trying to debug a bandwidth problem on a 3G connection, I tried speedtest.net, which ranks fairly high when one searches for “bandwidth test” on various search engines. I was getting very strange results, so I started wondering if my ISP might be bandwidth-throttling all traffic except the one from speedtest.net tests. After all, that’s on a 3G network, and another french 3G ISP (SFR) apparently uses Citrix ByteMobile to optimize the QoE by minifying HTML pages and recompressing images on-the-fly (amongst other things).

So, I fired wireshark, and discovered that no, it’s just speedtest being a bit naive. Speedtest uses its own text-based protocol on port 8080. Here is an excerpt of a download speed test:

> HI
< HELLO 2.1 2013-08-14.01
> DOWNLOAD 1000000
< DOWNLOAD JABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFG

Yeah, right: sequences of “ABCDEFGHIJ”. How course, extremely easy to compress, which apparently happens transparently on 3G (or is it PPP? but I tried to disable PPP compression, and it did not see any change).

It’s funny how digging into problems that look promising at first sight often results in big disappointments :-(

09 April 2014 à 19:23

19 March 2014

Alban Créquy

Traffic control for multimedia devices


Multimedia devices traditionally don't manage the network bandwidth required by applications. This causes a problem when users try to watch a streaming video or listen to a web radio seamlessly while other applications are downloading other content in the background. The background downloads can use too much bandwidth for the streaming video or web radio to keep up and users notice unnecessary interruptions in the playback.

I have been working on an approach to improve this using traffic control on Linux. This work was sponsored by Collabora.

What is traffic control


Traffic control is a technique to control network traffic in order to optimise or guarantee performance, low-latency, and/or bandwidth. This includes deciding which packets to accept at what rate in an input interface and determining which packets to transmit in what order at what rate on an output interface. 

On Linux, applications can send the traffic control configuration to the kernel using a Netlink socket with  the NETLINK_ROUTE protocol. By default, traffic control on Linux consists of a single queue which collects entering packets and dequeues them as quickly as the underlying device can accept them. The tc tool (from the iproute2 package) or the more recent "nl-*" tools (part of libnl) are different implementations but they can both be used to configure traffic control. Libnl has an incomplete support for traffic control but is in active development and progressing quickly.

Difficulty of shaping ingress traffic


Traffic control and shaping comes in two forms, the control of packets being received by the system (ingress) and the control of packets being sent out by the system (egress). Shaping outgoing traffic is reasonably straight-forward, as the system is in direct control of the traffic sent out through its interfaces. Shaping incoming traffic is however much harder as the decision on which packets to sent over the medium is controlled by the sending side and can't be directly controlled by the system itself.

However, for multimedia devices, control over incoming traffic is far more important then controlling outgoing traffic. Our use-case is ensuring glitch-free playback of a media stream (e.g. internet radio). In such a case, essentially, a minimal amount of incoming bandwidth needs to be reserved for the media stream.

For shaping (or rather influencing or policing) incoming traffic, the only practical approach is to put a fake bottleneck in place on the local system and rely on TCP congestion control to adjust its rate to match the intended rate as enforced by this bottleneck. With such a system it's possible to, for example, implement a policy where traffic that is not important for the current media stream (background traffic) can be limited, leaving the remaining available bandwidth for the more critical streams.

On Linux, ingress traffic control ("ingress qdisc" on the graph) happens before the Netfilter subsystem:
Netfilter-packet-flow
By Jengelh (Own work, Origin SVG PNG) [CC-BY-SA-3.0], via Wikimedia Commons



Difficulty of shaping on mobile networks


However, to complicate matters further, in mobile systems which are connected wirelessly to the internet and have a tendency to move around it's not possible to know the total amount of available bandwidth at any specific time as it's constantly changing. Which means, a simple strategy of capping background traffic at a static limit simply can't work.

The implemented solution


To cope with the dynamic nature, a traffic control daemon (tcmmd) has been implemented which can dynamically update the kernel configuration to match the current needs of the playback applications and adapt to the current network conditions. Furthermore to address the issues mentioned above, the implementation will use the following strategy:

Communication between the traffic control daemon and the applications is done via D-Bus. The  D-Bus interface allow applications to register critical streams by passing the standard 5-tuple (source ip and port, destination ip and port and protocol) which uniquely identify a stream and indicate when a particular stream bandwidth is too low.

To allow the daemon to effectively control the incoming traffic, a so-called Intermediate Functional Block device (ifb0) is used to provide a virtual network device to provide an artificial bottleneck. This is done by transparently redirecting the incoming traffic from the physical network device through the virtual network device and shape the traffic as it leaves the virtual device again. The reason for the traffic redirection is to allow the usage of the kernels egress traffic control to effectively be used on incoming traffic. The results in the example setup shown below (with eth0 being a physical interface and ifb0 the accompanying virtual interface).




To demonstrate the functionality as described above, a simple demonstration media application using Gstreamer (tcdemo) has been written that communicates with the Traffic control daemon in the manner described.

Testing, the set-up


The traffic control feature in tcdemo can be enabled or disabled on the command line. This allowed me to compare the behaviour in both cases. 

On my left, I have a web server serving both the files for a video stream and the files for background downloads. On my right, I have a multimedia device rendering a video stream while downloading other files on the same web server.

Traffic control is only useful when the available bandwidth is limited. In order to have meaningful tests, I simulated a low bandwidth with the following commands on the web server:
tc qdisc add dev wlan0 root handle 1: cbq avpkt 1000 bandwidth 10Mbit
tc class add dev wlan0 parent 1: classid 1:1 cbq rate 3Mbit allot 1500 prio 3 bounded isolated
tc filter add dev wlan0 parent 1: protocol ip u32 match ip protocol 6 0xff match ip sport 80 0xffff flowid 1:1

Only the traffic from port 80/http was limited. It is important to note that the background traffic and the stream traffic were both going through the same bottleneck.

Tcdemo was playing a video file streamed over http while 8 wgets were downloading the same file continuously. The 9 connections were competing for the limited bandwidth. Without traffic control, tcdemo would not have got enough bandwidth.

The following graph shows what happened with traffic control. The video streaming is composed of several phases:
  1. tcdemo opened the HTTP connection and its GStreamer pipeline started downloading. At the same time, tcmmd was notified there was a new stream connection and it restricted any potential background traffic to a very low limit. As long as the initial GStreamer queue was buffering, the background traffic limit did not change.
  2. The GStreamer queue became full at t=4s and the video started to be played on the screen. The daemon increased the limit on the background traffic exponentially and the stream bandwidth got reduced as a consequence.
  3. Despite the stream bandwidth degrading slowly, GStreamer managed to keep its queue over 75% full until t=25s. When the queue is more than 75% full, Gstreamer does not report it because tcdemo chose that threshold with the low-percent property on GstQueue2 (the graph shows 100% in this case). 
  4. At t=30s, the GStreamer queue was less than 70% full and that threshold triggered tcmmd to restrict the background traffic to its minimum.
  5. The stream could use most of the bandwidth and the GStreamer queue became full quickly at t=31s. The background traffic could start its exponential growth again.
traffic control stats
Thanks to traffic control, the GStreamer queue never got empty in my test.

Get the sources


git clone git://git.collabora.co.uk/git/user/alban/tcmmd
git clone https://github.com/alban/tcmmd

FAQ


Q: Do I need any privileges to run this?
A: No privileges required for tcdemo, the GStreamer application. But tcmmd needs CAP_NET_ADMIN to change the TC rules.

Q: The 5-tuple contains the TCP source port. How does the application know that number?
A: The application can either call bind(2) before connect(2) to choose a TCP source port, or call getsockname(2) after connect(2) to retrieve the TCP source port assigned automatically by the kernel. The former allows to install the traffic control rules before the call to connect(2) triggers the emission and reception of the first packets on the network. The latter means the first few packets will be exchanged without being shaped by the traffic control. Tcdemo implements the latter to avoid more invasive changes in the souphttpsrc GStreamer element and libsoup. See bgo#721807.

Q: What happens if an application forgets to unregister a 5-tuple when the video stream finishes?
A: That would be bad manners from the application. The current traffic control settings would remain.  And if the application notifies tcmmd that its buffer was empty and forgets to notify any changes, the background traffic would be severely throttled. However, if the application just terminates or crashes, tcmmd would notice it immediately on D-Bus and the traffic control rules would be removed.

Q: Does tcmmd remove its traffic control rules when terminated?
A: It depends how it is terminated. Tcmmd removes its traffic control rules on SIGINT and SIGTERM. But the rules remain in other cases (SIGSEGV, SIGKILL, etc.). If it is a problem in case of crash, tcmmd initialisation properly removes previous rules, so you could start tcmmd and interrupt it with ctrl-c.

Q: Instead of using the 5-tuple, why not using setsockopt-SO_MARK?
A: First, SO_MARK requires CAP_NET_ADMIN which is not something that media player should have. It could be worked around by fd-passing the socket to a more privileged daemon to call setsockopt-SO_MARK but it's not elegant. More importantly, tcmmd's goal is not to shape the egress traffic but the ingress traffic. The shaping of incoming packets is performed very early in the Linux network stack: it happens before Netfilter, and before the packet is associated to a socket. So we can't check the SO_MARK of a socket to shape incoming packets.

Q: Instead of using the 5-tuple, why not using cgroups?
A: The granularity of cgroups are only per-process. So the traffic control would not be able to distinguish between different HTTP connections in a web browser used to render a video stream and used for background downloads. And for the same reason as setsockopt-SO_MARK, it would not work for shaping ingress traffic: we would not be able to link the packet to any process or cgroup.

Q: Instead of sending the 5-tuple to tcmmd, why not set the IP type-of-service (TOS) on outgoing packets with setsockopt-SO_PRIORITY to avoid changes in the application and have an iptables target to feed that information about connections back to the ingress traffic control?
A: It could be possible if the bandwidth was fixed, but on mobile networks, the application needs to be changed anyway to give feedback when the queue in the GStreamer pipeline get emptied.

Q: Why not play with the TCP windows instead shaping the ingress traffic?
A: As far as I know, Linux does not have the infrastructure for that. The TCP windows to manipulate would not be from the GStreamer application but from all other connections, so it can't be done from userspace.

Q: Does tcdemo require any new feature in GStreamer?
A: Yes, souphttpsrc needs this patch: bgo#721807

Q: Does tcmmd require any new feature in the Linux kernel?
A: No.

Q: Does tcmmd work on several network interfaces (e.g. eth0 + wlan0)?
A: No, at the moment tcmmd only support one interface and it has to be started after the interface is up. Patches welcome!

Q: Tcmmd uses both libnl and /sbin/tc via system() calls. Why?
A: My goal is to use libnl and avoid spawning processes to call /sbin/tc. I just didn't have time to finish this. It will involve checking that libnl has the right features. Some needed features such as u32 action support were implemented recently in the last version.

Q: How did you get the graphs?
A: I used tcmmd's --save-stats option and the script tests/plot-tcmmd-log.sh.

Q: Why is there so frequent Netlink communication between tcmmd and the kernel?
A: One part of this is to gather regular statistics in order to generate graphs if the option --save-stats was used. The other part is for implementing the exponential progression of the bandwidth allocated to the background traffic: at regular interval, tcmmd changes the rate assigned to a qdisc. It could be avoided by implementing a specialised qdisc in the kernel for our use case. It would require more thinking how to design the API for that new qdisc.

Q: Does it work with IPv6?
A: No. The architecture is not specific to IPv4 but it is just not implemented yet for IPv6. Tcmmd would need to generate new TC rules because the IP headers are different between IPv4 and IPv6.


Thanks Sjoerd for the architecture diagram and proof-reading.

19 March 2014 à 14:11

27 February 2014

Lucas Nussbaum

self-hosting my calendar, follow-up

Following my blog post on the topic, I played a bit with various options.

But let’s explain my use case (which might be quite specific). I need to deal with three main sources of events:

Additionally, I follow some ICS feeds for some colleagues and other events.
I tend to access my calendar mostly on my computer, and sometimes on my N900 phone.

None of the web interfaces I looked at enabled me to (1) manage different calendars hosted on different CalDav servers; (2) subscribe to ICS feeds; (3) provide a CalDav interface to synchronize my phone.

I ended up using a radicale instance for my personal calendar, which was extremely easy to set up. It’s unfortunately a bit slow when there are many events (1600 since 2010 in my case), so I ended up importing only future events, and I will probably have to cleanup from time to time.

I switched to using IceDove with the Lightning add-on to manage all my calendars and ICS feeds. It’s unfortunately slower and less user-friendly than Google Calendar, but I’ll live with it.

On my N900, I used syncevolution to synchronize my various CalDav calendars. It works fine, but understanding how to configure it is rather tricky due to the number of concepts involved (templates, databases, servers, contexts, …). The synchronization is quite slow (several minutes for the 400-events Zimbra calendar), but works.

I also wanted a way to export my calendars to colleagues (both in a “free/busy” version, and in a “full information” version). I quickly hacked something using ruby-agcaldav (which is not packaged in Debian, and required quite a few dependencies, but it was easy to generate packages for all of them using gem2deb — the situation with other languages did not look better).
The resulting script is:


require 'agcaldav'
require 'date'

cal = AgCalDAV::Client.new(:uri => 'LABCALDAVSERVER', :user => 'xx', :password => "xx")
ev = cal.find_events(:start => '2014-02-01', :end => '2200-01-01')

cal = AgCalDAV::Client.new(:uri => 'RADICALESERVER', :user => 'xx', :password => "xx")
ev2 = cal.find_events(:start => '2014-02-01', :end => '2200-01-01')

limit = (Time::now - 7*86400).to_datetime

# create new empty calendars
ncpriv = Icalendar::Calendar.new
ncpub = Icalendar::Calendar.new

(ev + ev2).each do |e|
next if e.end < limit # drop old events to keep the calendar small

# build event for the free/busy calendar
pe = Icalendar::Event.new
pe.start = e.start
pe.end = e.end
pe.klass = "PRIVATE"
pe.transp = e.transp
ncpriv.add(pe)

# build event for the calendar with event information
pube = Icalendar::Event.new
pube.start = e.start
pube.end = e.end
pube.transp = e.transp
if not e.klass == "PRIVATE"
pube.summary = e.summary
pube.location = e.location
end
ncpub.add(pube)
end

# export free/busy calendar
fd = File::new('xx.ics', 'w')
fd.puts ncpriv.to_ical
fd.close

# export calendar with event information
fd = File::new('yy-Zeeh9bie.ics', 'w')
fd.puts ncpub.to_ical
fd.close

So, mostly everything works. The only thing that doesn't is that I haven't found a way to subscribe to an ICS feed on my N900. Any ideas?

27 February 2014 à 16:39

25 February 2014

Lucas Nussbaum

self-hosting my calendar

I’m trying to self-host my calendar setup, and I must admit that I’m lost between all the different solutions.

My requirements are:

It does not seem to be possible to find a single framework doing all of the above. AFAIK:

What did I miss?

25 February 2014 à 21:31

17 January 2014

Lucas Nussbaum

RSS feed available in the Debian Maintainer Dashboard

One of my pet projects in Debian is the Debian Maintainer Dashboard. Built on top of UDD, DMD provides a maintainer-centric view to answer the “I have a few hours for Debian, what should I do now?” question (see example).

Christophe Siraut did a lot of great work recently on DMD, rewriting much of the internals. As a result, he also added a RSS feed feature: you can now get notified of new TODO list items by subscribing to that feed.

If you have suggestions or comments, please use the debian-qa@ list (see this thread).

Thanks, Christophe!

17 January 2014 à 09:20

07 October 2013

Lucas Nussbaum

talk at Open World Forum

I attended Open World Forum last week (thanks to Inria for funding my travel). It was a fantastic opportunity to meet many people, and to watch great talks. If I had to single out just one talk, it would clearly be John Sullivan’s What do you mean you can’t Skype?!.

On Saturday, I delivered a talk presenting the Debian project. It was my first DPL-ish talk to the general public, so it still needs some tuning, but I think it went quite well (slides available). Next opportunity to talk about Debian: LORIA, Nancy, France, 2013-10-17 13:30 (iPAC seminar).

07 October 2013 à 14:33

25 September 2013

Lucas Nussbaum

Software for brainstorm / ideas sharing and voting?

Some time ago, Ubuntu had Ubuntu Brainstorm, a website where non-developers could submit ideas of improvements, and other people could comment and vote on them. I was wondering if there was existing software to deploy a similar service, e.g. as a plugin to popular forum software. I’ve found ideascale.com, but relying on the Cloud for that is not acceptable for my planned use.

(For clarification: my immediate interest for that is unrelated to Debian work)

25 September 2013 à 07:33

21 August 2013

Lucas Nussbaum

DebConf was fantastic (not just the view)

So, I’m back from DebConf, which probably translated to the 10 busiest days of my life, but also to one of the best times of my life. The Le Camp venue clearly contributed to this success: having everybody at the same place, but at the same time many opportunities for quiet chat or just enjoying the view, was really a good idea. Everybody who made DebConf possible deserve a huge “thank you”, as well as all attendees: it is really a honor to be a part of such a fantastic community.

Now, let’s go back to daily life, and to my re-filled Debian TODO list!

21 August 2013 à 07:21

16 August 2013

Lucas Nussbaum

Debian birthday

This upload was the first one of my very first package in Debian. It was sponsored by Dafydd Harries, who I’ve finally met at DebConf13, and got out of NEW on 2005-08-16. Exactly 8 years ago today. I only realized that yesterday evening, and Debian’s birthday feels even more special to me now. Dafydd, it looks like I owe you a lot! :)

16 August 2013 à 07:27

DebConf

This DebConf is obviously quite special for me, being the DPL. It has really been great so far to talk to meet so many people, and especially to meet so many new Debian contributors or Developers. I’m really happy to see that the next generation is ready! :)

On Sunday, I delivered my Bits from the DPL talk. The video is available, and I’ve finally uploaded the slides (working link here, it seems that Penta ate my slides). I hope you enjoyed it/will enjoy it!

16 August 2013 à 07:20

03 August 2013

Olivier Crête

GNOME.Asia 2013

This June, I was in Seoul, Korea for the GNOME.Asia Summit, the yearly occasion to meet up with the Asian side of the GNOME community. As always, it was an awesome conference, with so many cool people. I learned about new projects like Seafile and got to meet new friends and catch up with old ones.

I’d also to thank my employer, Collabora, for sponsoring my flight and the GNOME foundation for paying the hotel.

Sponsored by Collabora                              Sponsored by the GNOME Foundation

03 August 2013 à 21:49

28 July 2013

Vincent Untz

From Thessaloniki with love -- openSUSE Conference 2013

Last week-end I was in Greece, in Thessaloniki, enjoying the openSUSE Conference 2013. If I had to summarize the event in one word, that would be: wow! It was the first time we had this event in another city than Nuremberg and Prague (two places where SUSE has offices), and it was the first time the organization was fully lead by the community. I was quite confident that things couldn't go wrong since, after all, what matters is that we're all in the same place. But I was amazed that the whole event went so smoothly! This was really a great job from a whole team of volunteers:

oSC13 volunteers

Just to give an example of the hard work that was accomplished: most (all?) talks were successfully streamed, and the recordings are already online! Stella and Kostas definitely deserve credits for the overall success, as they kept leading the organization in the right direction since last year, and the event wouldn't have been possible without their dedication. Our sponsors also helped make all this happen, so many thanks to SUSE, ARM, DevHdR and Oracle!

Having people from all over the world was once again an opportunity to meet up with old and new friends, who were coming from Brazil (Izabel, Carlos), the US, all over Europe obviously, but also India (Manu, Saurabh) and China as well as Taiwan (Sunny, Max, David, etc.)... The conference is the global event of the openSUSE community, without any doubt. With 250 attendees, there were a lot of hallway chats and informal meetings; I'm sure the GNOME couch tradition that we initiated with Dominique and Richard will stay over the years ;-)

oSC13 volunteers

Unsurprisingly, the openSUSE Board took opportunity of having so many community members to discuss several topics with as many people as possible. The board also organized for the first time a session about team reports. Even though several teams didn't participate to that session (generally because no team members was there), we had more than ten teams joining the party on stage, and this was probably one of the best way to see how broad our community really is and to learn the latest developments in various areas of the project. We also had our usual town hall meeting which went rather nicely, with useful feedback to the board.

oSC13 volunteers

The bad thing for me is that I had to stay only for a few days due to work, but there's already a next opportunity to meet with the community: this will be the openSUSE Summit in Orlando next November. And if you cannot make it, then I can only recommending making sure that you will join us next year, for the openSUSE Conference in Dubrovnik!

oSC13 volunteers

28 July 2013 à 17:35

01 July 2013

Vincent Untz

OpenStack Rhône-Alpes Meet-up #1

Last Thursday was the first meet-up of the OpenStack Rhône-Alpes group, in Grenoble. The idea of organizing such a meet-up came up just one month ago, at Solutions Linux; having such an OpenStack meet-up in Rhône-Alpes feels so natural, given the amount of people we have in the area (we can even include Geneva ;-)). Dave did a great job with the organization, and HP provided a pretty nice venue (and sponsored food!).

First OpenStack Meet-up in Rhône-Alpes!

We managed to attract around 25 people to this event, and given that it was our first and that we did nearly no noise about it, it's not too bad :-) We had people attending from HP, Bull, Red Hat, SUSE and more, and four of us delivered quick talks about XLcloud (by Patrick Petit, from Bull), oVirt (by Dave Neary, from Red Hat), HP's public cloud (by Gavin Brebner, from HP) and Crowbar (by yours truly). I must admit that Gavin Brebner's presentation about abusing HP's public cloud was extremely interesting; there's a lot of cool stuff happening to keep a public cloud running.

Of course, we enjoyed some nice food afterwards and stayed chatting a bit longer about OpenStack and other technical bits. And best of all: we celebrated OpenStack's third birthday a bit early:

Happy birthday OpenStack!

If everything goes well, the next meet-up of the group will be in Lyon, in September, stay tuned! And we'll make more noise, so we hope more people will join!

01 July 2013 à 19:24

13 May 2013

Vincent Untz

Mangée par la Poste

Visiblement, la Poste utilise des machines qui, parfois, ont faim et doivent se nourrir :

Mangée par la Poste

Dommage pour cette lettre !

13 May 2013 à 15:16

30 March 2013

Lucas Nussbaum

Debian releases used by popcon participants

The graph below is generated from popcon submissions. Since they include the version of the popularity-contest package, one can determine the Debian release that was used by the submitter (a new version of the popularity-contest package is generally uploaded just after the release to make that tracking possible).

The graph is similar to the one found on popcon, except that versions newer than the latest stable release are aggregated as “testing/unstable”.

Comments:

Scripts are available on git.debian.org.

30 March 2013 à 13:52

16 March 2013

Lucas Nussbaum

Ideas from the -vote@ DPL election discussions

After one week of campaign on -vote@, many subjects have been mentioned already. I’m trying here to list the concrete, actionable ideas I found interesting (does not necessarily mean that I agree with all of them) and that may be worth further discussion at a less busy time. There’s obviously some amount of subjectivity in such a list, and I’m also slightly biased ;) . Feel free to point to missing ideas or references (when an idea appeared in several emails, I’ve generally tried to use the first reference).

On the campaign itself, and having general discussions inside Debian:

On getting new users and contributors to Debian:

Infrastructure, processes, releases:

Relationships with upstreams/downstreams:

This list could be moved to wiki.d.o if others find sufficiently useful to help maintaining it.

16 March 2013 à 19:05

13 March 2013

Vincent Untz

openSUSE 12.3 is out, with OpenStack love

Get openSUSE 12.3!

Have you heard about it? openSUSE 12.3 is out!

I did an upgrade earlier today on my main laptop (with a simple zypper dup after having updated the repos configuration, which went surprisingly fast), and this release looks great! But the best part: it comes with OpenStack love!

Enjoy Folsom!

For the first time, an openSUSE release provides a fully working set of OpenStack packages. We had some OpenStack packages in the previous release, but they were not in such a great shape and some components were even missing (although we fixed that later on with packages in the build service).

With 12.3, you can finally enjoy OpenStack with the Folsom release in a very straight-forward way, and therefore you can easily deploy your own cloud. The packages that we provide are built from the stable/folsom branch, and there's an online update coming soon so you can enjoy the stable/folsom code as of end of last week.

To help people who might not want to learn everything needed to properly deploy OpenStack, we also have a small openstack-quickstart package, that comes with a script that can be used to deploy everything locally. It is obviously not recommended to run this on your main computer (I usually run this in a virtual machine), but it gets you quickly to the point where you can play with OpenStack.

OpenStack Folsom on openSUSE 12.3

Dashboard of an OpenStack cloud running on openSUSE 12.3

Play today with Grizzly!

Of course, Folsom is relatively old at this point and the new version, Grizzly, is to be released in three weeks. But don't be sad! We've been working on Grizzly packages for some time now: you can grab them from the Cloud:OpenStack:Master project in the build service (hey, look, it's even building packages for SLE and openSUSE 12.2! the build service is a rather convenient tool!). I guess we'll properly move them to Cloud:OpenStack:Grizzly once Grizzly is officially released.

Develop with DevStack!

I mentioned a few months ago that I had finished porting DevStack to openSUSE 12.2, and I wrote some small documentation on how to use it. It's really a neat tool, both for playing with OpenStack and for developing for it.

However, I realized earlier this week that I had never double-checked everything was still okay for 12.3. It turns out there's a small issue that completely breaks it, oops ;-) But once the fix is checked in, DevStack will be usable on the latest openSUSE. I'll do some more tests before marking this version of openSUSE as supported in DevStack, but that shouldn't block anyone from using DevStack on 12.3.

Join us!

We're pretty open about how we develop OpenStack in openSUSE. Andreas wrote a post about all this a few days ago. We've opened up (or rather, revived) a mailing list dedicated to the cloud recently, which developers, packagers and users can all use to discuss OpenStack. And unsurprisingly, we also have an #opensuse-cloud channel on Freenode. But most importantly, we've worked on making public the infrastructure we use to build OpenStack for openSUSE.

I think the important bit on this is that everybody is able, and welcome, to join this effort. It's not just about being able to say "see, we have OpenStack"; it's about building a rock-solid experience for OpenStack, and enjoying doing that!

Now, let's celebrate the release: party time! :-)

13 March 2013 à 19:02

04 March 2013

Lucas Nussbaum

Debian is (still) changing

(Looking for those graphs online, I realized that I never properly published them, besides that old post)

I’ve been playing with snapshot.d.o, which is a fantastic resource if you want to look at Debian from an historical perspective (well, since 2005 at least).

Team maintenance

comaint
We now have more team-maintained packages than packages maintained by someone alone. Interestingly, the “small, ad-hoc group of developers” model does not really take off.

Maintenance using a VCS

vcs

A large majority of our packages are maintained in a VCS repository, with Git being the clear winner now.

Possible goal for Jessie: standardize on a Git workflow, since every team tends to design its own?

Packaging helpers

helpers

Again, we have a clear winner here, with dh. It’s interesting to note that, while dh was designed as a CDBS killer, it kind-of fails in that role.

Possible goal for Jessie: deprecate at least pure-debhelper packaging?

Patch systems and packaging formats

formats-patches

Again, clear winner with 3.0 (quilt).

The (dirty) scripts that generate those graphs are available in Git (but you need to connect to stabile to execute them, and it’s rather time consuming — hours/days).

04 March 2013 à 13:05

03 March 2013

Lucas Nussbaum

DPL game

When Francesca started her DPL game, I too started think about possible candidates. Here is my shortlist of dream candidates:

Seriously, if you are a DD, you have the right to run. There’s no need for someone to nominate you. If you think that you could possibly say something interesting during the discussion period, and can spare the time to participate in the -vote@ discussions, please run. DPL campaigns used to be a great time where Debian visions, goals, politics and random stuff were discussed. The more candidates, the more interesting campaigns (8 candidates in 2007!).

Also, there are already two three other candidates, so even if you don’t want the job, it’s not that risky.

(Initially, I thought about nominating everyone, but security-wise, it might not be such a great idea.)

03 March 2013 à 19:54

01 February 2013

Vincent Untz

Next stop: FOSDEM 2013

In a couple of hours, I'll be taking the train and heading to Brussels for FOSDEM. I've lost counts of how many FOSDEM I've attended, which is probably a good indication of how great the event is!

FOSDEM

As usual, this will be a good place to catch up with friends, but also to talk with tons of different people about so many topics. If you want to chat about OpenStack, SUSE Cloud, openSUSE or GNOME, I'll be glad to join you.

The schedule is quite packed, but from what I can tell so far, I'll be sitting in the cloud devroom on Sunday (don't hesitate to join in order to learn about what's happening in the OpenStack world!). Oh, I'll also give a talk in Janson about challenges that the GNOME project is facing, just before the closing keynote.

And no, I won't have my blue hat, so you'll need to find another way to catch me (hint: I have a SUSE backpack nowadays) ;-)

01 February 2013 à 10:56

25 January 2013

Vincent Untz

And here comes a gnome-panel fork...

Last week-end, just before leaving for some travel, I became aware that gnome-panel was being forked into consort-panel (btw, I commented on that post, but I guess it was a bit too late since it's stuck in the moderation queue).

Now, let me start by stating clearly that I have nothing against forks: people are free to go this way, and that's cool with me. However, I quickly got confused for three reasons: I thought it was clear that volunteers are welcome to maintain gnome-panel, I thought I had explained to Ikey in June 2012 why some changes would be blocked from entering fallback mode but could hopefully happen in a not-too-distant future, and I'm getting explicitly blamed here and there for putting roadblocks.

I usually don't mind being blamed, but I prefer when it's for good reasons ;-) Of course, as a maintainer, I reject patches. There are usually good reasons, including the fact that there's a design philosophy that a module like gnome-panel had to follow since it was fully part of GNOME. Rejecting patches is part of the maintainer job. It doesn't mean that contributions are not welcome, but I guess it can be perceived as such... Another task of a maintainer is to enable people to keep the code alive, and in the case of gnome-panel, it was clear to me that having the fallback mode as part of GNOME 3 was a blocker to do so. It took more time than I would have liked, but this is something that got fixed when the fallback mode got dropped of GNOME 3.

With this in mind, and to clarify why I got confused by the fork announcement, here's a quick timeline of events in 2012, related to the fate of gnome-panel, covering what I was aware of until the blog post from a few days ago:

The ironic point here, at least to me, is that it's Ikey's mail that triggered my push for the fallback mode to be dropped from official GNOME so people could work on gnome-panel with more freedom. Which is what seems to be wanted.

Anyway, let me take this as an opportunity to remind everyone that people are welcome to become maintainers of gnome-panel. It'd be preferable to maintain it in the GNOME infrastructure, but I guess even just forking it with full history on gitorious/github would work. No need to rename, no need to follow the GNOME 3 design, etc. If full forking+renaming is preferred for some reason, in the end, that's fine; I'd be curious to know what good reason exists, though.

And as usual, you're welcome to blame me for X, Y or Z :-)

25 January 2013 à 15:08

21 December 2012

Vincent Untz

DevStack on openSUSE, or how to quickly setup OpenStack on openSUSE

One of the first things I did when I joined the Cloud team at SUSE was to start porting DevStack to openSUSE. DevStack is a set of shell scripts to build complete OpenStack development environments. It is useful to create a small OpenStack environment that will be used for hacking, testing, etc. and is therefore primarily used for upstream development. Getting this to work on openSUSE seemed like a logical first step before doing more OpenStack work. I got things working pretty quickly, but for various reasons, this all stayed in a personal branch of mine (except for a few preliminary patches).

A few weeks ago, I got time for DevStack again. So I rebased my branch, cleaned up everything, and started submitting the patches. After reviews from Sean Dague and Dean Troyer (and some patch rewriting to address the issues that were raised), the openSUSE support landed in master. A few other people tested it, and nobody has been hitting any major issue, so yesterday, I finally submitted the patch to make openSUSE 12.2 a supported distribution. Now you can play with DevStack on openSUSE!

I wrote some documentation for DevStack on openSUSE if you want to get more details on how to use it. But I guess it wouldn't hurt to show how easy it is to setup your own OpenStack environment this way...

I'm shamelessly stealing instructions from the single VM DevStack guide to show you the very short version. Just run the following in an openSUSE virtual machine (do not run this on your main system unless you're 100% sure it's what you want: DevStack is a little bit too invasive right now; see Daniel's analysis on this topic):

zypper in git-core
git clone https://github.com/openstack-dev/devstack.git
cd devstack
echo ADMIN_PASSWORD=password > localrc
echo MYSQL_PASSWORD=password >> localrc
echo RABBIT_PASSWORD=password >> localrc
echo SERVICE_PASSWORD=password >> localrc
echo SERVICE_TOKEN=tokentoken >> localrc
echo FLAT_INTERFACE=br100 >> localrc
./stack.sh

(You'll actually need to call FORCE=yes ./stack.sh until the patch mentioned above gets in.)

And there you go, you have OpenStack running! That was quite easy, right? :-) You can connect to the web dashboard (horizon) or use the command line tools (hint: source openrc will setup the proper environment variables for you). Here's a few commands you can use to get started:

source openrc
glance image-list # find out which image is available
nova boot --image cirros-0.3.0-x86_64-uec --flavor m1.tiny cirros-test # start an instance of one specific image
nova list # see what instances are in the cloud

So go ahead, read the documentation, play with all this, and enjoy DevStack on openSUSE!

21 December 2012 à 13:02

27 November 2012

Vincent Untz

JDLL and Mini-DebConf Paris 2012

During the last two week-ends, I went to two different events. That's part of my end-of-year sprint where I travel too much: SUSEcon and openSUSE Summit in September, OpenStack Summit and openSUSE Conference in October (oops, didn't find time to write about these events), two weeks vacation in Thailand in October/November (yes, we enjoyed the time there!), one week of team meeting in Prague right now, and two other trips to Paris during those few months... Crazy planning!

I attended these events with my advocate hat to deliver GNOME-related talks (and also to chat with people a bit about openSUSE, and of course to meet good friends of mine ;-)). I feel there's a big need on GNOME's side to communicate more and clarify our direction and opinions, and on top of that, there's a lot of mis-informed statements around that people blindly trust and that need to be debunked. My talks were simply part of my local contribution towards that goal. And apparently, that's something that seems to be most welcome!

JDLL 2012

The Journées du Logiciel Libre (or JDLL) is an event that occurs every year in Lyon. Lyon being close to home, it's an event I can attend quite easily and this is not something I can complain about ;-) We did have some great people at the event this year, including a french-turned-british-turned-french-again guy.

When I got asked to give a talk about GNOME this year, I wasn't sure I would have anything really interesting to tell, so I suggested an interactive session around the recent hot topics in GNOME (you know, GNOME OS, systemd, fallback mode, etc.). In the end, even though I had many slides ready, we simply discussed the questions that were raised by the audience, and I believe that this session proved to be very useful for the attendees. So a good experience, and a format I'll likely use again.

I also had the opportunity to play a bit with Firefox OS. I've been following the project for quite some time but never took time to really try it, so I was really glad to be able to take a long look at it. There's still some work to do, and, hrm, well, that was visible ;-) I managed to crash things without even trying to be nasty. I hope it will take off, though: there's a need for an alternative closer to our ideals.

Mini-DebConf Paris 2012

The Debian France team organized a Mini-DebConf in Paris, and I was invited for a slot. I chose to talk about GNOME vs downstreams, and discuss the love/hate relationship we have, and how the future direction can be good/bad for different downstreams. The idea was simply to get out some information out about what GNOME is doing, and to clarify where the project is heading, as this has some pretty big impact on our downstream friends. Obviously not everything is perfect in GNOME but I feel that the project is, overall, doing okay as an upstream. (I'm kind of sad to discover an ABI breakage in glib after I told to Stefano and Lucas that we were not breaking ABI in our platform; oh well).

This Mini-DebConf was a pleasant surprise, as there were quite a number of attendees, and the whole event went quite smoothly (well, at least for the day I was there). It was also interesting to hear about the different opinions with regards to the Debian release cycle (got some pretty good food for thoughts), and I enjoyed Sylvestre's talk about making Debian compiler agnostic. The event had many other great talks — definitely an event I'd recommend attending, even to non-Debian people.

27 November 2012 à 23:15

25 November 2012

Lucas Nussbaum

Half of the package maintainers are not DDs or DMs

During the Paris Mini-Debconf, Nicolas Dandrimont talked about The state of mentors.debian.net: GSoC and beyond. He said that Half of Debian’s packages are maintained by sponsored maintainers. That statement was actually wrong, as he confirmed later.

However, using a few UDD queries, I could come up with:

Full UDD notes:

all packages in sid:
select source, version from sources_uniq where release = 'sid'

packages in sid known to upload_history:
select source, version from upload_history where
(source, version) in (select source, version from sources_uniq where release = 'sid')

packages that were uploaded by the changed_by person:
create temporary table sources_not_sponsored as select distinct source, version
 from upload_history, carnivore_keys, carnivore_emails
 where (source, version) in (select source, version from sources_uniq where release = 'sid')
 and fingerprint = key
 and carnivore_keys.id = carnivore_emails.id
 and carnivore_emails.email = changed_by_email;

packages not uploaded by the changed_by person:
create temp table uh_sid as select source, version, fingerprint, changed_by_email
from upload_history
where (source, version) in (select source, version from sources_uniq where release = 'sid');

create temp table uh_sid_sponsored as select source, version, fingerprint, changed_by_email from uh_sid
where (source, version) not in (select source, version from sources_not_sponsored);

list with sponsor login:
select distinct source, version, fingerprint, changed_by_email, login
from uh_sid_sponsored
left join carnivore_keys on fingerprint = key
left join carnivore_login on carnivore_keys.id = carnivore_login.id;

=> 4188 sponsored packages. some of them are in a strange state (changed_by is a DD, but uploaded by another DD). excluding those:

create temp table sponsored_but_dds as select distinct source, version, fingerprint, changed_by_email, login
from uh_sid_sponsored, carnivore_emails, carnivore_login
where changed_by_email = carnivore_emails.email
and carnivore_emails.id = carnivore_login.id;

create temp table really_sponsored as select distinct source, version, fingerprint, changed_by_email, login
from uh_sid_sponsored
left join carnivore_keys on fingerprint = key
left join carnivore_login on carnivore_keys.id = carnivore_login.id
where (source, version) not in (select source, version from sponsored_but_dds);

=> 3147 sponsored packages

select distinct changed_by_email from really_sponsored ;
=> 963 different sponsorees

select distinct changed_by_email from upload_history where
(source, version) in (select source, version from sources_uniq where release = 'sid');
=> 2015 distinct emails.

no DD amongst maintainer or uploader:

create temp table dds_emails as select email from carnivore_emails, carnivore_login
where carnivore_emails.id = carnivore_login.id;

select source, version, maintainer, uploaders from sources_uniq
where release='sid'
and maintainer_email not in (select * from dds_emails)
and not exists (select * from uploaders where release = 'sid' and sources_uniq.source = uploaders.source and sources_uniq.version = uploaders.version and email in (select * from dds_emails))
and maintainer_email != 'packages@qa.debian.org'
and (source, version) in (select source, version from really_sponsored);

25 November 2012 à 23:31