Our client – the Public Health Institute - is a body under Ministry of Health Malaysia. The institute performs significant research on public health matters and conducts country wide surveys every year gathering data related to public health services and the perception of Malaysian population about these services. Data scientists at our client organization then analyze this data and present to public policy experts.
The health survey is a set of very elaborate questionnaires for different segments of the population (children, adolescents, adults, married/un-married, smokers/non-smokers and several other categories).
For years, the survey was conducted on printed questionnaires. Each questionnaire was about 500 questions with fairly complex survey rules that each surveyor was trained to follow. Our client would have to train several hundred surveyors every year over an extensive month-long training program about the survey rules and flow of the questionnaires. The team of surveyors would then go all over Malaysia in a door to door campaign filling out the stack of printed questionnaires. The campaign takes 3 months on average.
Once the surveys were filled out, there was a massive exercise of keying in all the data into a database where our client’s team of data scientists would make use of the precious data. This keying in of data was error-prone, extremely labor intensive and would take months for the data to be available to the relevant team members at our client organization.
GoodCore Software designed a large-scale Android app along with its associated backend that completely replaced the paper based questionnaires and our client conducted its 2015 survey based on that app. This helped achieved significant advantages for our client, namely the following
The app necessitated some advance design and architecture considerations. These are highlighted below.
Anyone familiar with the nature of software development understands that no amount of testing can completely eliminate the possibility of a bug 100% in a large-scale mobile app. We were well aware of that and therefore a fairly elaborate mechanism was put in place in the architecture of the app to ensure that, in-case of the highly undesirable bug, we were automatically notified about the specifics of the bug and the associated usage scenario that caused the bug – without expecting the end user to provide all these details. This enabled our team to take corrective actions quickly while the 2015 survey was in progress.
One of the challenges of the app design was the fact that survey team responsible for conducting the surveys would typically carry the tablets with the app to far-flung areas of Malaysia where there was often no internet. A sophisticated offline mode was built into the app which allowed for a seamless experience to the user. The data would automatically synchronize with the server when the internet connection as available.
The app communicates with the server side database using a secure services layer that enforces user roles and encrypts data for security. Our client spends months preparing the survey and significant resources on conducting the survey so the data captured in the process is very valuable. The app and the backend was hardened against common security threats like SQL injection and cross site scripting.
Since live data was being recorded, having any serious bug in the app had serious implications – it would potentially waste weeks of effort if errors were found in recording the data. There was no way the surveyors would go back to the respondent of a survey to “correct” any data. This necessitated a very elaborate quality assurance and regression testing plan to harden the app well before it was handed out for live surveys. GoodCore’s testing team worked relentlessly to make sure this happens and the hard work of the survey team is not lost.
Considering that each survey was a lengthy set of data points, and each tablet would contain potentially hundreds of survey records on local storage, we had to make optimum use of storage space and the data access had to be optimized for a quality user experience.
We implemented a real time reporting tool for our client to monitor the progress of the surveys in 2015 across Malaysia. In addition to the new reporting tool, we also integrated the data into the legacy Business Intelligence (BI) tools used by our for further data crunching and scientific analysis.