Category: Software and data processing

Some months ago, we launched the awtiCode core team. Now, awtiCode has offered its first Python training: Basic Python for Environmental Dataprocessing. In previous years I gave Python trainings to colleagues of the Arba Minch Water Technology Institute (AWTI). I am very happy that from these trainings and the launch of awtiCode, through awtiCode colleagues have now stood up to offer the training by themselves. The training was fully offered by Awel Haji, Bahafta Gebresilassie, Beyene Senedu, Demiso Daba and Israel Gebresilasie.

Python training practice

Certificates

Python training

Approximately twenty participants followed the training, divided over two separate groups. The training focused on the use of Python for Environmental Data Processing. It moved from the very basics (variables, collections, functions) to applied topics (satellite data, visualizing data on a map). In total, the training took five weeks of two full mornings per week. Training materials were earlier created by Nimrod de Wit, and slightly revised by the trainers. You can find all training materials on my GitHub repository.

Together with colleagues from the Arba Minch Water Technology Institute (AWTI), I have launched awtiCode: Python code by and for AWTI staff. With a core team, we are planning bi-weekly meetings, where progress and plans are discussed. We will inquire dataprocessing needs amongst colleagues, and (try to) code solutions. Code is hosted on GitHub repositories: /jddingemanse/awticode and /awticode.

Current members of the awtiCode core team are Awel Haji, Bahafta Gebresilassie, Beyene Senedu, Daniel Asele, Demiso Daba, Israel Gebresilasie, Mesele Markos and Nebiyu Waliyi. The launch of awtiCode is another step in promoting Python dataprocessing in my institute. I have given various trainings, such as a Basic Python training (see here and here), and a Training of Trainers. True programming skills, however, come from doing. The assumption is that awtiCode will result in coding experience for the core members.

My brother in law, Nimrod de Wit, came to Arba Minch to live and work with us for some months. He has quite some experience with Python. Based on this experience, our visitor offered a training on Basic Python for Environmental Data Processing to my colleagues of the Arba Minch Water Technology Institute. 31 persons attended one or more training sessions. 17 persons received certificates for attending all sessions and submitting an assignment in which they used Python for their own data.

I have offered a Python training to AWTI staff already multiple times (march 2021, june 2021, october 2021 and october 2022). Nimrod fully revised those materials. My trainings leaned heavily on Powerpoint slides and simultaneous small practice tasks in the Spyder IDE. Nimrod shifted to small (30 minute) introductions, after which participants could work by themselves through Juypter Notebooks. The training materials are on my GitHub repository.

I was glad to be the assistant rather than the trainer during a training for once. I am very glad for the revised training materials, which force the participants to practice much more. A training might help people to get started, but skills in something like programming are ultimately from a lot of hours of simply trying and using it.

I offered a Training of Trainers on Basic Python for Environmental Data Processing. Participants were selected from the four faculties of the Arba Minch Water Technology Institute (AWTI), as well as the Water Resources Research Centre (WRRC). The intention was to create in each of these faculties a Python team of 2-3 staff members that can provide courses and support within their respective faculty. Eleven participants received their certificate during a closing ceremony. During that ceremony, each participant presented their plan for what they would do with Python for AWTI. Scientific directors of both AWTI and the Arba Minch Institute of Technology (AMIT) were present as well.

I have offered a Python training for AWTI staff already three times (in March 2021, June 2021 and October 2021). Approximately 100 colleagues have come to one or more training sessions. The number of staff really taking up Python, however, is low. Our newest idea was therefore to offer a focused Training of Trainers (TOT). Participants were asked to provide a motivation at the start of the training, and write a plan on what they will do with Python for AWTI after the training. Also, the training materials were extended a bit, and spread out over more sessions. The earlier Python trainings took five days – or ten mornings. This TOT took ten full days. The increased time was filled with a bit of new material, but especially more time for practice.

I have shared the training materials on an OSF repository.

On request of Dr. Ashenafi Hailu, I constructed and offered a training in Basic Python for Biology Data Processing. 12 PhD students participated in the training. Daniel Asele assisted me during the training. The training took five full days, all within one week.

For this training, I followed the general outline of the training I gave earlier for environmental data: module 1 and 2 as the absolute basics (variables, conditions, loops, etcetera), and modules 3 and 4 for working with data (Pandas) and visualizing data (Matplotlib). I based data examples on the field of biology. The fifth module went into some applications more specific to biology: some statistics, (sequence) string manipulation, and working with the package BioPython. I have shared all training materials on an OSF repository.

This training was a (probably one-time-only) move out of my normal field. It cost me quite some time to get acquainted with terminology and techniques for biology. This spent time directly translated itself into more experience. However, in the future I prefer to stick to tools and knowledge used in my own field (air quality). Or, that of my direct colleagues (Arba Minch Water Technology Institute; environmental sciences).

Apart from that, this training was an experiment with offering it within one week (five days in a row). For a next training, I will shift back to one or two sessions per week. Five days in a row made it impossible for participants to practice with module topics before moving on to a next module.

The Ethiopian Meteorology Institute (EMI) collects meteorological data from several weather stations all over Ethiopia. Data is processed manually in Microsoft Excel. To validate this data, or to produce tables or graphs, employees spend much time. Zerihun Bikila, employee of EMI and a master student in one of my Python classes, came up with the idea to create automated data processing tools with Python, under the name ‘PyCAMT’ (Python based Climate Data Processing and Visualization). Based on this, we worked on the development of a python package for meteorological data in Ethiopia.

For a month, I have enthusiastically coded together a preliminary package for importing, processing and visualizing EMI station data. Zerihun worked on a user interface in Jupyter Notebook. We tried to pitch the package to EMI management, but this so far proved difficult. Anyways, it was good practice for me to create a package in Python, and I am sure that I will use what I learned in the near future to help people automize and professionalize data processing and visualization.

I hosted the Python code as pycamtET on my GitHub repository. The Jupyter Notebook interface is hosted as pycamtETinterface. Below, you can see slides that introduce the package.

The first 15 GMB laptops arrived in Ethiopia by mail. While already promised in March and transported in May, finally they are in the lab, and students can start to use them!

The laptops were sent by post, but once arrived in Ethiopia, it was quite a challenge to get them to Arba Minch. While the resale value per laptop was 150 euro, the laptops were taxed as new. Furthermore, initially a penalty of 300% was awarded because it was seen as importing laptops without license. After multiple visits to Addis Ababa and a lot of paperwork, the penalty was waived and the laptops were forwarded to Arba Minch. The remaining tax will be refunded by Arba Minch University. Thanks to them and thanks to GMB my students can now get practical computer classes which are not affected by power outages and absent lab assistants!

GMB promised to donate in total 30 laptops. Based on the experience with the first batch, the next batch of 15 laptops will be transported by people visiting us from The Netherlands.

Many colleagues of Arba Minch Water Technology Institute (AWTI) showed interest in attending the Python training which I offered last March. I therefore offered another round of the same training. You can find the training materials through this link.

Based on the previous round, I made a couple of changes:

• I introduced practice sessions. Experience from the previous round showed that participants mainly followed the training sessions, but did not practice much by themselves. By offering assignments and sessions for working on those assignments, I hoped to stimulate personal practice.
• Training assistants joined the sessions. From the earlier training I invited colleagues to help during the training. In this way, there were multiple persons available for personal support of the participants. I was assisted by Daniel Asele, Getachew Enssa, Manyazwal Getachew and Samrawit Dereje.
• I included the possibility of earning a certificate. Earlier, I was unaware that a big reason for joining trainings is to get a certificate. I do however not believe in handing out a certificate based on only training attendance. Therefore, participants need to make a final assignment to be eligible for a certificate, by using their own data.

Interest versus application

The interest in the training was high (66 registrations). However, similar to the previous round, there was quite a difference between interest and actively participating in the full training. 54 persons attended at least one session, but 32 of these attended most sessions for the four core modules. At the moment, only the fifth (bonus) session on multidimensional (satellite) data remains to be offered. Apart from participating in the training, there is also the gap between following a training and using it yourself. Many participants do not realize that skills in Python depend on personal application much more than on following a training. Initially I thought that I could help 60+ persons towards better data processing. I have to adjust my expectations, and I must be happy with the five to ten colleagues that truly get started on using Python.

The past week I have offered a training on Advanced Excel for data processing. Experts in several fields world-wide use Microsoft Excel. In scientific circles, programming languages like Python, Matlab or R are often mentioned. However, Microsoft Excel is encountered a lot in the work field. Advanced use of Microsoft Excel (wildcards, array formulas, nested formulas, assigning variable names, etcetera) enables data processing on a professional level. I gave a training in Python recently. An increased knowledge in Excel helps colleagues as well.

Due to high interest in the training (68 registrations), I offered the two-day training in two shifts. One of the shift was even in two simultaneous sessions. Daniel Asele, Samrawit Dereje and Geertje Dingemanse assisted me in offering the sessions. A small part of the training consisted out of one-way communication with slide presentations. The main part of the training required participants to work themselves through worksheets with step-by-step instructions, under supervision of the instructors.

I divided the training into five modules: assigning names, conditions and searching, grouping data, array formulas and miscellaneous topics. The training materials are available on an OSF repository.

Over a period of five weeks, I have offered a five-day Python training for environmental data processing. You can find the training materials on my OSF repository.

During the year 2020, I found myself with a lot of time (students were sent home due to corona) and data (during air pollution courses I offered, students collected data across 30 situations). I decided to spend my time on learning to use Python for data processing. [UPDATE: this resulted in data visualizations for an article about the students’ data collection.] To foster my newfound Python addiction, I decided to turn what I learned into a training for colleagues of the Arba Minch Water Technology Institute. This resulted in a ‘Basic Python for Environmental Data Processing’ training. The training ran from the absolute basics (variables, functions) up to data processing and visualization (Pandas, Matplotlib) and working with multidimensional data (NetCDF).

The training leaned heavily on lecture slides and participants simultaneously practicing on their own laptop. I offered four modules and one bonus-day, distributed over five weeks (one day per week). To keep groups small, I offered a singe module four times to four groups of participants. While the interest in Python appears to be high (48 participants at the start), the motivation to actively participate in all sessions lacks behind (17 participants followed all sessions). I will have to find out in future trainings whether that is something I can influence.