Data Export

Regular data exporters (1) collect objects from specific containers, (2) iterate over the collected objects, (3) extract and mangle information from each object, (4) write the information of each object into a row of a CSV file and (5) finally provide the file for download. The CSV file is neither stored in the database nor archived in the filesystem. (3) and (4) means a flattening of the hierarchical data structure, i.e. a mapping of objects to flat relational data to be stored in a CSV table. The extracted information must not necessarily be based only on static attributes of the collected object. The data, finally stored in the CSV file, can also be derived from parent or child objects, or dynamically computed by the object’s methods and property attributes. These methods and properties can retrieve information from everywhere in the portal’s database. In the following we list all exporter classes including two attributes and a method description. The fields attribute contain the column titles of the export file. These are not necessarily only attributes of the exported objects.

Note

The list of exported columns usually underlies heavy customizations. In the Kofa base package only very few columns are being exported. In some Kofa custom packages tons of data are being gathered from applicants and students and the number of columns increases accordingly.

The title attribute unveils the name of the exporter under which this exporter will be displayed in the user interface. The mangle_value() method shows how some of fields are being dynamically computed.

Regular Exporters

User Exporter

class waeup.kofa.userscontainer.UserExporter[source]

The User Exporter exports all user accounts. It iterates over all objects of the users container.

fields = ('name', 'title', 'public_name', 'description', 'email', 'phone', 'roles', 'local_roles', 'password')
title = u'Users'
mangle_value()[source]

The mangler determines the local roles each user has and computes a Python expression like:

{u'waeup.local.ClearanceOfficer': [u'faculties/ABC/', u'faculties/DEF/']}

Faculty Exporter

class waeup.kofa.university.export.FacultyExporter[source]

The Faculty Exporter exports all faculties in the faculties container. This is the only place where faculties are stored.

fields = ('code', 'title', 'title_prefix', 'users_with_local_roles', 'officer_1', 'officer_2')
title = u'Faculties'
mangle_value()[source]

The mangler computes the users_with_local_roles value which is a Python expression like:

[{'user_name': u'bob', 'local_role': u'bobsrole'}, {'user_name': u'anna', 'local_role': u'annasrole'}]

Department Exporter

class waeup.kofa.university.export.DepartmentExporter[source]

The Department Exporter exports all departments stored in the faculty containers. The exporter iterates over all faculties and then over all departments inside each faculty container.

fields = ('code', 'faculty_code', 'title', 'title_prefix', 'users_with_local_roles', 'officer_1', 'officer_2', 'officer_3', 'officer_4')
title = u'Departments'
mangle_value()[source]

The mangler additionally computes the faculty_code value which is the code (= object id) of the faculty that hosts the department.

Course Exporter

class waeup.kofa.university.export.CourseExporter[source]

The Course Exporter exports all courses in the database. It iterates over all departments and faculties.

fields = ('code', 'faculty_code', 'department_code', 'title', 'credits', 'passmark', 'semester', 'users_with_local_roles', 'former_course', 'results_validated_by', 'results_validation_date', 'results_validation_session')
title = u'Courses'
mangle_value()[source]

The mangler additionally computes the department_code and faculty_code values.

Certificate Exporter

class waeup.kofa.university.export.CertificateExporter[source]

The Certificate Exporter exports all certificates in the database. It iterates over all departments and faculties.

fields = ('code', 'faculty_code', 'department_code', 'title', 'study_mode', 'degree', 'start_level', 'end_level', 'application_category', 'ratio', 'school_fee_1', 'school_fee_2', 'school_fee_3', 'school_fee_4', 'custom_textline_1', 'custom_textline_2', 'custom_float_1', 'custom_float_2', 'users_with_local_roles')
title = u'Certificates'
mangle_value()

The mangler additionally computes the department_code and faculty_code values.

Certificate Course Exporter

class waeup.kofa.university.export.CertificateCourseExporter[source]

The Certificate Course Exporter exports all certificate courses (CertificateCourse instances) in the database. It iterates over all departments and faculties.

fields = ('course', 'faculty_code', 'department_code', 'certificate_code', 'level', 'mandatory', 'course_category')
title = u'Courses in Certificates'
mangle_value()[source]

The mangler computes the codes of the faculty, the department and the certificate which require the course. It also exports the course code.

Note

The course must not necessarily be offered by the same department.

Access Code Batch Exporter

class waeup.kofa.accesscodes.export.AccessCodeBatchExporter[source]

The Access Code Batch Exporter exports container data. It does not export access codes inside the container.

fields = ('cost', 'creation_date', 'creator', 'disabled_num', 'entry_num', 'num', 'prefix', 'used_num', 'batch_id')
title = u'Access Code Batches'

Access Code Exporter

class waeup.kofa.accesscodes.export.AccessCodeExporter[source]

The Access Code Exporter exports all access codes stored in the access code batch containers. The exporter iterates over all access code batches and over all access codes inside each batch container.

fields = ('batch_num', 'batch_prefix', 'batch_serial', 'cost', 'history', 'owner', 'random_num', 'representation', 'state')
title = u'Access Codes'
mangle_value()[source]

The mangler adds a hash symbol at the end of random_num to avoid annoying automatic number transformation by Excel or Calc.

Hostel Exporter

class waeup.kofa.hostels.export.HostelExporter[source]

The Hostel Exporter exports container data. It does not export beds inside the container.

fields = ('beds_for_all', 'beds_for_final', 'beds_for_fresh', 'beds_for_pre', 'beds_for_returning', 'blocks_for_female', 'blocks_for_male', 'floors_per_block', 'hostel_id', 'hostel_name', 'maint_fee', 'rooms_per_floor', 'sort_id', 'special_handling')
title = u'Hostels'

Bed Exporter

class waeup.kofa.hostels.export.BedExporter[source]

The Bed Exporter exports all beds stored in the hostel containers. The exporter iterates over all hostels and over all beds inside each hostel container.

fields = ('bed_id', 'bed_number', 'bed_type', 'owner', 'hall', 'block', 'room', 'bed', 'special_handling', 'sex', 'bt')
title = u'Beds'

Document Exporter

class waeup.kofa.documents.export.DocumentExporterBase[source]

This is the exporter base class for all kinds of document exporters. Derived classes export all documents of type class_name inside the documents container.

All schema fields and the property attributes history, state, class_name and users_with_local_roles are being exported.

mangle_value()[source]

The mangler prepares the history messages and computes the users_with_local_roles value which is a Python expression like:

[{'user_name': u'bob', 'local_role': u'bobsrole'}, {'user_name': u'anna', 'local_role': u'annasrole'}]

Application Data Exporters

Applicants Container Exporter

class waeup.kofa.applicants.export.ApplicantsContainerExporter[source]

The Applicants Container Exporter exports container data. It does not export applicants (application records) inside the container.

fields = ('application_category', 'application_fee', 'application_slip_notice', 'code', 'description', 'enddate', 'hidden', 'mode', 'prefix', 'send_email', 'startdate', 'strict_deadline', 'title', 'with_picture', 'year')
title = u'Applicants Containers'

Applicant Exporter

class waeup.kofa.applicants.export.ApplicantExporter[source]

The Applicant Exporter exports application records (= applicants) stored in the database. In contrast to the exporters in the academic section this exporter does not iterate over the items of containers but searches the ApplicantsCatalog instead.

The exporter exports all applicants if started in the Data Center which means in the context of the DataCenter object. The exporter can also be started ‘locally’ which means in the context of an ApplicantsContainer container. Then the export_filtered() instead of the export_all() method is applied which searches for applicants in the respective container.

fields = ('applicant_id', 'course1', 'course2', 'course_admitted', 'date_of_birth', 'email', 'firstname', 'lastname', 'locked', 'middlename', 'notice', 'phone', 'referees', 'reg_number', 'school_grades', 'sex', 'special_application', 'student_id', 'suspended', 'password', 'state', 'history', 'container_code', 'application_number', 'display_fullname', 'application_date')
title = u'Applicants'
mangle_value()[source]

The mangler determines the codes of the atributes course1, course2 and course_admitted. It furthermore prepares the history messages and adds a hash symbol at the end of the phone number to avoid annoying automatic number transformation by Excel or Calc.

Applicant Payment Exporter

class waeup.kofa.applicants.export.ApplicantPaymentExporter[source]

The Applicant Payment Exporter exports all payments made by applicants. In other words, it exports payment tickets in state ‘paid’. The exporter searches ApplicantsCatalog and iterates over all payment tickets which are stored in an applicant (container).

The exporter exports all applicant payments if started in the Data Center which means in the context of the DataCenter object. The exporter can also be started ‘locally’ which means in the context of an ApplicantsContainer container, see ApplicantExporter above.

fields = ('ac', 'amount_auth', 'creation_date', 'p_category', 'p_combi', 'p_id', 'p_item', 'p_session', 'p_state', 'payment_date', 'r_amount_approved', 'r_code', 'r_desc', 'applicant_id', 'reg_number', 'display_fullname')
title = u'Applicant Payments'
mangle_value()[source]

The mangler determines the applicant’s id.

Student Data Exporters

When starting a Student Data Exporter in the Data Center all student records will be taken into consideration, no matter what or where a student is studying. The exporter can also be started ‘locally’ at various levels in the academic section. Starting one of the exporters e.g. at faculty or department level means that only the data of students are exported who study in this faculty or department respectively. The exporter can also be started at certificate level. Then only the data of students, who are studying the named study course, will be taken into account. At course level the data of those students are being exported who have attended or taken this specific course.

Student Data Exporter can be further configured through a configuration page. Search parameters like the student’s current level, current session and current study mode can be set to filter sets of students in order to decrease the size of the export file. The set of filter parameters varies and depends on the ‘locatation’ from where the exporter is called. A completely different set of filter parameters is provided for courses. In this case the session and level can be selected when the course was taken by the student.

Student Exporter

class waeup.kofa.students.export.StudentExporter[source]

The Student Exporter first filters the set of students by searching the students catalog. Then it exports student base data of this set of students.

fields = ('adm_code', 'clr_code', 'date_of_birth', 'email', 'employer', 'firstname', 'flash_notice', 'lastname', 'matric_number', 'middlename', 'nationality', 'officer_comment', 'parents_email', 'perm_address', 'personal_updated', 'phone', 'reg_number', 'sex', 'student_id', 'suspended', 'suspended_comment', 'password', 'state', 'history', 'certcode', 'is_postgrad', 'current_level', 'current_session', 'entry_session')
title = u'Students (Data Backup)'
mangle_value()[source]

The mangler prepares the history messages and adds a hash symbol at the end of the phone number to avoid annoying automatic number transformation by Excel or Calc.

Student Study Course Exporter

class waeup.kofa.students.export.StudentStudyCourseExporter[source]

The Student Study Course Exporter first filters the set of students by searching the students catalog. Then it exports the data of the current study course container of each student from this set. It does not export their content.

fields = ('certificate', 'current_level', 'current_session', 'current_verdict', 'entry_mode', 'entry_session', 'previous_verdict', 'student_id', 'previous')
title = u'Student Study Courses (Data Backup)'
mangle_value()[source]

The mangler determines the certificate code and the student id.

Student Study Level Exporter

class waeup.kofa.students.export.StudentStudyLevelExporter[source]

The Student Study Level Exporter first filters the set of students by searching the students catalog. Then it exports the data of the student’s study level container data but not their content (course tickets). The exporter iterates over all objects in the students’ studycourse containers.

fields = ('gpa', 'level', 'level_session', 'level_verdict', 'total_credits', 'transcript_remark', 'validated_by', 'validation_date', 'student_id', 'number_of_tickets', 'certcode', 'previous')
title = u'Student Study Levels (Data Backup)'
mangle_value()[source]

The mangler determines the student id, nothing else.

Course Ticket Exporter

class waeup.kofa.students.export.CourseTicketExporter[source]

The Course Ticket Exporter exports course tickets. Usually, the exporter first filters the set of students by searching the students catalog. Then it collects and iterates over all studylevel containers of the filtered student set and finally iterates over all items inside these containers.

If the course code is passed through, the exporter uses a different catalog. It searches for students in the course tickets catalog and exports those course tickets which belong to the given course code and also meet level and session passed through at the same time. This happens if the exporter is called at course level in the academic section.

fields = ('automatic', 'carry_over', 'code', 'course_category', 'credits', 'dcode', 'fcode', 'level', 'level_session', 'mandatory', 'outstanding', 'passmark', 'score', 'semester', 'ticket_session', 'title', 'student_id', 'certcode', 'display_fullname', 'previous')
title = u'Course Tickets (Data Backup)'
mangle_value()[source]

The mangler determines the student’s id and fullname.

Student Payment Exporter

class waeup.kofa.students.export.StudentPaymentExporter[source]

The Student Payment Exporter first filters the set of students by searching the students catalog. Then it exports student payment tickets by iterating over the items of the student’s payments container. If the payment period is given, only tickets, which were paid in payment period, are considered for export.

fields = ('ac', 'amount_auth', 'creation_date', 'p_category', 'p_combi', 'p_current', 'p_id', 'p_item', 'p_level', 'p_session', 'p_state', 'payment_date', 'r_amount_approved', 'r_code', 'r_desc', 'student_id', 'state', 'current_session')
title = u'Student Payments (Data Backup)'
mangle_value()[source]

The mangler determines the student’s id, registration state and current session.

Bed Ticket Exporter

class waeup.kofa.students.export.BedTicketExporter[source]

The Bed Ticket Exporter first filters the set of students by searching the students catalog. Then it exports bed tickets by iterating over the items of the student’s accommodation container.

fields = ('bed', 'bed_coordinates', 'bed_type', 'booking_code', 'booking_date', 'booking_session', 'student_id', 'actual_bed_type')
title = u'Bed Tickets (Data Backup)'
mangle_value()[source]

The mangler determines the student id and the type of the bed which has been booked in the ticket.

Note

The above exporters refer to a specific content type (object class). They export all attributes of these objects and a few additional parameters derived from the parent objects. These exporters can be used for reimport, or more precisely for backing up and restoring data. The following ‘special’ exporters are made on request of some universities to collect and compose student data for analysis and postprocessing by the university.

Outstanding Courses Exporter

class waeup.kofa.students.export.OutstandingCoursesExporter[source]

The Student Outstanding Courses Exporter first filters the set of students by searching the students catalog. Then it exports students with lists of outstanding courses, i.e. courses which the student has missed (not registered at all), failed (registered but not passed) or nottaken (registered but not taken).

fields = ('student_id', 'matric_number', 'certcode', 'display_fullname', 'missed', 'failed', 'nottaken')
title = u'Outstanding Courses'
mangle_value()[source]

The mangler determines the student’s id, fullname and certcode, and it collects the lists of outstanding courses.

Data For Bursary Exporter

class waeup.kofa.students.export.DataForBursaryExporter[source]

The Data for Bursary Exporter works just like the Student Payment Exporter but it exports much more information about the student. It combines payment and student data in one table in order to spare postprocessing of two seperate export files. The exporter is primarily used by bursary officers who have exclusively access to this exporter. The exporter exports paid, waived and scholarship payment tickets.

fields = ('ac', 'amount_auth', 'creation_date', 'p_category', 'p_combi', 'p_current', 'p_id', 'p_item', 'p_level', 'p_session', 'p_state', 'payment_date', 'r_amount_approved', 'r_code', 'r_desc', 'student_id', 'matric_number', 'reg_number', 'firstname', 'middlename', 'lastname', 'sex', 'state', 'current_session', 'entry_session', 'entry_mode', 'faccode', 'depcode', 'certcode')
title = u'Payment Data for Bursary'
mangle_value()[source]

The mangler fetches the student data.

Unpaid Payments Exporter

class waeup.kofa.students.export.UnpaidPaymentsExporter[source]

The Unpaid Payments Exporter works just like the Student Payment (singular intended) Exporter but it exports only unpaid tickets. This exporter is designed for finding and finally purging outdated payment tickets.

title = u'Unpaid Payment Tickets'

Accommodation Payments Exporter

class waeup.kofa.students.export.AccommodationPaymentsExporter[source]

The Accommodation Payments Exporter works like the Data for Bursary Exporter above. The exporter exports paid, waived and scholarship payment tickets with category bed_allocation or hostel_maintenance. The exporter is primarily used by accommodation officers who have exclusively access to this exporter.

fields = ('ac', 'amount_auth', 'creation_date', 'p_category', 'p_combi', 'p_current', 'p_id', 'p_item', 'p_level', 'p_session', 'p_state', 'payment_date', 'r_amount_approved', 'r_code', 'r_desc', 'student_id', 'matric_number', 'reg_number', 'firstname', 'middlename', 'lastname', 'sex', 'state', 'current_session', 'entry_session', 'entry_mode', 'faccode', 'depcode', 'certcode')
title = u'Accommodation Payments'
mangle_value()

The mangler fetches the student data.

School Fee Payments Overview Exporter

class waeup.kofa.students.export.SchoolFeePaymentsOverviewExporter[source]

The School Fee Payments Overview Exporter first filters the set of students by searching the students catalog. Then it exports some student base data together with the total school fee amount paid in each year over a predefined year range (current year - 9, ... , current year + 1).

fields = ('student_id', 'matric_number', 'display_fullname', 'state', 'certcode', 'faccode', 'depcode', 'is_postgrad', 'current_level', 'current_session', 'current_mode', 'entry_session', 'reg_number', '2013', '2014', '2015', '2016', '2017', '2018', '2019', '2020', '2021', '2022', '2023', '2024')
title = u'School Fee Payments Overview'
curr_year = 2024
mangle_value()[source]

The mangler summarizes the school fee amounts made per year. It iterates over all paid school fee payment tickets and adds together the amounts paid in a year. Waived payments are marked waived and scholarship payments marked scholarship.

Session Payments Overview Exporter

class waeup.kofa.students.export.SessionPaymentsOverviewExporter[source]

The Session Payments Overview Exporter first filters the set of students by searching the students catalog. Then it exports some student base data together with the total amount paid in predefined payment categories over the previous three session (referring to current academic session). Sample output:

header: ...schoolfee13,schoolfee14,schoolfee15,gown13,gown14,gown15...

data: ...2000.0,,3000.0,,,1000.0,...

This csv data string means that the student paid 2000.0 school fee in 2013 and 3000.0 in 2015. S/He furthermore paid 1000.0 for gown rental in 2015.

fields
title = u'Session Payments Overview'
mangle_value()[source]

Study Levels Overview Exporter

class waeup.kofa.students.export.StudyLevelsOverviewExporter[source]

The Student Study Levels Overview Exporter first filters the set of students by searching the students catalog. Then it exports some student base data together with the session key of registered levels. Sample output:

header: ...100,110,120,200,210,220,300...

data: ...2010,,,2011,2012,,2013...

This csv data string means that level 100 was registered in session 2010/2011, level 200 in session 2011/2012, level 210 (200 on 1st probation) in session 2012/2013 and level 300 in session 2013/2014.

fields = ('student_id', 'state', 'certcode', 'faccode', 'depcode', 'is_postgrad', 'entry_session', 'current_level', 'current_session', '0', '10', '100', '110', '120', '200', '210', '220', '300', '310', '320', '400', '410', '420', '500', '510', '520', '600', '610', '620', '700', '710', '720', '800', '810', '820', '900', '910', '920', '999', '1000')
title = u'Study Levels Overview'
mangle_value()[source]

The mangler checks if a given level has been registered. It returns the level_session attribute of the student study level object if the named level exists.

Combo Card Data Exporter

class waeup.kofa.students.export.ComboCardDataExporter[source]

Like all other exporters the Combo Card Data Exporter first filters the set of students by searching the students catalog. Then it exports some student base data which are neccessary to print for the Interswitch combo card (identity card for students). The output contains a passport_path column which contains the filesystem path of the passport image file. If no path is given, no passport image file exists.

fields = ('display_fullname', 'student_id', 'matric_number', 'certificate', 'faculty', 'department', 'passport_path')
title = u'Combo Card Data'
mangle_value()[source]

The mangler determines the titles of faculty, department and certificate. It also computes the path of passport image file stored in the filesystem.

Data For Lecturer Exporter

class waeup.kofa.students.export.DataForLecturerExporter[source]

The Data for Lecturer Exporter searches for students in the course tickets catalog and exports those course tickets which belong to the given course code, meet level and session passed through at the same time, and which are editable by lecturers (disabled on 10/03/21). This exporter can only be called at course level in the academic section.

fields = ('matric_number', 'student_id', 'display_fullname', 'level', 'code', 'level_session', 'score')
title = u'Data for Lecturer'
mangle_value()[source]

The mangler determines the student’s id and fullname.

Transcript Data Exporter

class waeup.kofa.students.export.TranscriptDataExporter[source]

The Transcript Data Exporter first filters the set of students by searching the students catalog. Then it exports student data along with their transcript data.

fields = ('student_id', 'state', 'certcode', 'faccode', 'depcode', 'entry_session', 'current_level', 'current_session', 'transcript_data')
title = u'Transcript Data'
mangle_value()[source]

The mangler determines and formats the transcript data.

File Export

You want to export files (passport images, pdf slips), for instance of all students in a department or applicants in an applicants container? There is no facility in Kofa which does this job for you, but you can batch-download files by means of your operating system on your local machine. This works perfectly with the the wget command on Linux or MacOS computers. Windows does not provide such a command.

If numbers.txt contains the application numbers of applicants in the applicants container xyz, the following bash script will download all passport images directly onto you computer:

wget --save-cookies cookies.txt --keep-session-cookies --post-data 'form.login=my-username&form.password=my-password' https://my-kofa-site/login

for i in $(cat numbers.txt)
do
 wget --load-cookies cookies.txt --output-document=$i.jpg https://my-kofa-site/applicants/xyz/$i/passport.jpg
done

If numbers.txt contains the ids of students in a department, the following bash script will download all passport images directly onto you computer. The script is a slightly extended, more user friendly version of the script above:

if [ $# -lt 3 ]
then
   echo "usage: $0 username password filename"
   exit 1
fi

wget --save-cookies cookies.txt --keep-session-cookies --post-data "form.login=$1&form.password=$2" https://my-kofa-site/login

counter=0

while read variable
do
   counter=$((counter+1))
   wget --load-cookies cookies.txt --output-document $variable.jpg https://my-kofa-site/students/$variable/passport.jpg
   if [[ "$counter" -gt 1000 ]]; then
      wget --save-cookies cookies.txt --keep-session-cookies --post-data "form.login=$1&form.password=$2" https://my-kofa-site/login
      counter=0
   fi
done < $3