28 January, 2015

Android Decompilation Testing


  
What’s an APK file?
Android application package (APK) is the package file format to install application software on Google’s Android operating systems. APKs are similar to installers where code is first compiled and its parts are packaged into one file. An APK file contains program’s code (*.dex files), manifest file, resources, assets and certificates.

An APK is simply an archive file with a set of classes and resources. The compiled Dalvik executable bytecode will be available in classes.dex file. Classes.dex file contains compiled executable code - classes & methods having business logic for problem being solved. One can convert classes.dex file (which is in machine readable format) to JAR file (which is human readable) with the help of tools available for this purpose. Relevant tools are listed in appendix for further reference.

Organized Structure of APK File
The file structure of an APK file looks as below:

Specific files related to the app like fonts, 3rd party SDKs and integration of it can be found in com folder.

Manifest file
An Android app must have “AndroidManfiest.xml” file in APK’s root folder as seen in the file structure above. It includes permissions needed by the app while it is installed on the device, activities and other details. There are Android apps which helps to view manifest of an installed app. Manifest file can also be parsed and read from APK file using appropriate tools (listed in Appendix).

Dex (Dalvik Executable)
Android apps are written in Java and executed in DVM (Dalvik Virtual Machine). This is different from JVM (Java Virtual Machine).  DVM is built for Android OS.  The Java bytecode is transformed into DVM bytecode and stored as .dex (Dalvik Executable) and .odex (Optimized Dalvik Executable) files. The terms odex and deodex is associated with Dalvik bytecode operations.

When Java compiler has produced JVM bytecode, Dalvik compiler deletes .class files and recompiles them into Dalvik bytecode. After this, Dalvik compiler merges this into one .dex file. During this process, interpretation, translation and reconstruction happens in class definitions, function code executed by target VM, method names, reference names, etc.

There is a limitation on what .dex file can include and how much it can. Due to this, APK can have one .dex file or more than one .dex file associated with it.
A hint about this limit can be obtained from the JAR obtained during conversion of .dex to JAR and viewing in JAVA decompiler. This tells about the complexity of the app and how to plan tests across multiple .dex files associated with the app for better coverage. With Android 5.0’s ART, dex file is converted to OAT file in this way è Java -> Class à Dexà OAT. Analysis of this can lead to devising better tests.

JAR Decompilation
JAR file contents can be viewed using Java decompiler.  Usually, the decompiled content in bytecode will be obfuscated by programmers. This gives visibility into classes and methods. While this does not comprise entire source code, it does provide hints with respect to the business logic used in programming the app. Reading through source code surely helps testers to identify what tests can be done and what recommendations can be given to the development team.
If the code is not obfuscated, it has to be brought to the attention of product owners and programmers.  Obfuscation is possible using tools like ProGuard and DexGuard.

Key Tests to be performed on a decompiled JAR file
1. Readable Business Logic
Presence of obfuscated code indicates the developer may have put some security mechanism in place. If the code is in plain text format, it means the code is readable and understandable to an extent, providing visibility to compiled functions and classes. Some part of this code might be patented already or about to be patented. And, this part of code may have the core logic of the app. Keeping the code in the open in logically readable format, may incur heavy loss to the organization in revenues and reputation. 

Solution
Code must be obfuscated at all times. Obfuscation is a procedure where organization masks source code to avoid others from viewing or copying the code/algorithms to build similar apps. ProGuard is one of the tools that can be used by programming team to obfuscate code.

2. Manifest file
Permissions are easily visible to anyone who views manifest file, hence giving an overview of what kind of permissions are needed for the app. This is again an unsafe option.

Solution
It is advisable that manifest files are encrypted using methods that will make it hard for the hacker to decrypt and see. In web technologies, a few encryption algorithms are used. In mobile, you may have to explore if there are similar algorithms or techniques for encryption or obfuscation. [Android store guidelines insists that permissions are visible to users. It’s upto testers to review these permissions used and identify potential risks]

3. getDeviceID() usage
It is fair for anyone to track individual installations on specific devices on a diverse user base. It sounds plausible just to call TelephonyManager.getDeviceId() and use that value to identify the installation. There are problems with this: Firstly, it doesn’t work reliably. Secondly, when it does work, that value survives device wipes (“Factory resets”) and thus one could end up making a nasty mistake when one of the customers wipes their device and passes it on to another person.

Solution
There are many good reasons for avoiding an attempt to identify a particular device. For those who want to try, the best approach is probably the use of ANDROID_ID on anything reasonably modern, with some fallback heuristics for legacy devices.

It is recommended to study the operation of permissions and activities in the app. If these are used and not handled well, the app may allow using the data which it processes on the device. The decompiled JAR can be of help here.

4. Accessibility to Resource files
Resource XMLs, important images and other files may be easily downloadable and directly consumable by anyone who does so. It is important to secure these files.

Solution
Images should not be accessible for anyone post de-compilation process of the APK.

Summary
It is a common myth that JAR decompilation is related to Security Testing. However, it is not true. Decompiling helps to learn the app better. One can learn how functionality works; understand the performance of the app and possible areas where the memory usage is a concern for the app.  Next time you decompile an app, look at the treasure trove in front of you and explore innumerable possibilities.
My team and I have applied above key tests for a few android apps we tested. It gave good insight to developers about the app itself and how it is perceived in the google world. Some of them who didn’t know much about the threats associated with APK files learned a lot from this exercise. Additionally, it also helped my team build credibility with the development team.

Token of Thanks
My community colleague Ravisuriya reviewed this article thoroughly and supporting me patiently in my learning. Thanks Ravi, for your precious time.

P.S: I cannot share the actual APK report format for obvious confidentiality reasons, however, this article gives a short overview of different things that can be done with decompiled JARs.

Appendix [for additional reading]
What’s the need to review java code?
       1.       Basic Health Check of the app (via java source code and other confidential data)
       2.       To perform security audits (provided code is not obfuscated)
    1. Sensitive Data Exposure
    2. Inadequate Access Controls
    3. Code Injection Vulnerabilities
    4. Broken Authentication and Session Management
       3.       Reverse engineering third-party protocols or APIs

APK Protection
Protecting an APK file is an area to be concerned about in this free internet era. Developers must have an eye on this while building APK files. There are many methods to protect APK files. There are several articles that brief you about them. I have listed out high level headers here, so you can go read them up in detail on Android developer site
  1. Secure Java source code
  2. Encrypting class files
  3. Code Obfuscation
Tools that might help in the process of JAR decompilation
          1.       Dex2jar – Tool to work with android *.dex and java *.class files
          2.       JAD – Java decompiler is a decompiler for Java programming language
          3.       JD-GUI – JD-GUI is a graphical utility that displays source code of *.class files
          4.       APKtool – It is a third party reverse engineering tool for android apps
          5.       Winzip / WinRAR – Compression/decompression utility
          6.       Proguard- ProGuard is a free Java class file shrinker, optimizer, and obfuscator. 
           7.       APK Protect – This is an APK protection solution

References
http://www.decompileandroid.com/
Image Credits - World Wide Web



14 January, 2015

Device Fragmentation – How to Tame The Bull


This article was originally published in Testing Trapeze December 2014 Edition.

As per the 2013 IDC report, mobile internet traffic is growing 1.5x per year and likely to grow much faster in future. Social media usage across the world shows a growing participation on mobile devices. Players like Snapchat, Instagram, Pinterest and Facebook are having more mobile users than on the web. This data suggests that many software organizations/developers/users are switching to building/using mobile apps more than ever. This, now, brings us to the question of how to get these mobile apps tested on millions of devices. In this article, I will share my thoughts and also highlight how I applied Jonathan Kohl’s approach to solve the device fragmentation problem and also build a mobile test strategy.

Device Fragmentation

Mobile device fragmentation occurs when some users are running older versions of the operating system, while other users are using newer versions. Some even call it Device Diversity.
The term mobile device fragmentation is also used to describe different versions of the same operating system that are created when an original equipment manufacturer (OEM) modifies an open source mobile operating system for specific products.  



Device fragmentation could arise due to various reasons:

Different Platform
Organizations assume that fragmentation needs to be addressed on Android alone as it is home to several OS version and mobile device types. However, with newer platforms showing up, the fragmentation bug is biting iOS, Windows and newer platforms too. Even testing on iOS with multiple device models and operating systems is becoming cumbersome although it’s not as complex as testing on Android yet. Note the word “yet.”

Hardware Dependency
Several features on apps are hardware dependent and hence need devices from specific manufacturers on particular device makes/models. For instance, Android 2.3 has support for Near Field Communications (NFC), but many phones do not have NFC hardware.

Manufacturer Customizations
In the Android world, manufacturers usually take a version of the operating system as a baseline and add their customizations and ideas, thus putting modern versions in the phone. iOS doesn’t pose this threat yet, but as it loosens some of its guidelines over a period of time, it is not a far off challenge on iOS too.

CPU/Memory Footprint Limitations
Many apps are memory/CPU intensive which means they perform well on high end phones and struggle to cope well on phones with limited capabilities. This is one of the top challenges of fragmentation for app developers today.

How to you choose from an ocean of devices?

Device usage is an incredibly important factor to consider. There are millions of devices in the market. How do you decide which ones to choose optimally in terms of time, cost, and effort? This is a universal question several organizations would like an answer for.



This picture is the best way of visualizing the number of different Android devices that have downloaded the OpenSignal app in the past few months. According to OpenSignal developers, the fragmentation has tripled from previous year. If you are building an app that has similar fragmentation, how would you address that?

Solving the device fragmentation problem starts with seeking answers to questions like:
1.       What platforms are customers using?
2.       What are the operating system versions?
3.       What are the multiplicity of screen sizes and resolutions used?
4.       Which are the manufacturers of mobile devices, makes, and models?
5.       What are the most popular mobile devices on the market?
6.       What are the most frequently used mobile browsers?
7.       Which devices demonstrate more problems to mobile apps compared to others?
8.       Which new devices should an already released mobile app support?
9.       How many previous versions of the platform should a mobile app support?

And the list of questions goes on.

Passing through the above list of questions is only the beginning. Listing out answers to most of them only brings more questions. However, as complex as it might sound, answers to these questions can help stakeholders narrow down the number/type of mobile devices used for testing.
In his book, “Tap into Mobile Application Testing”, author Jonathan Kohl lists out 4 main approaches that help in choosing a test strategy for mobile apps. I will highlight those 4 approaches below:

Singular Approach

Testing is done on one device type. Some organizations build apps that are focused on a specific mobile device. For example, a mobile app developed for viewing the menu and placing a food order at restaurants is sure to be installed on a specific tablet from a specific manufacturer of a particular version of the operating system. In this case, the stakeholders make a conscious choice of which device to go with and build an app just for that device. This is a straight forward case where testing on a single device will be good enough.
Some stakeholders might consciously make a choice of supporting just one device to start with and hence focus only on one device.

Proportional Approach

Testing is done on multiple devices. “How many devices are good enough?” is a question that can be answered best by doing basic research on mobile traffic and analytics data. Suppose the app has 80% Android mobile traffic and 15% iOS mobile traffic and the remaining traffic comes from other devices. This information helps in focusing most testing efforts on the Android platform for most part and *some* testing on the iOS platform. This is best suited for mobile apps that are already out in the market and there is visibility on the incoming traffic and mobile analytics.

Shotgun Approach

Shotgun means ‘as many as possible’. In this approach, Testing is done on multiple devices on a very large scale. This is best suited to test mobile apps where stakeholders don’t place any restrictions on mobile devices/platforms used. With hundreds of platforms and platform versions, it’s extremely hard to test on all devices to cover these. There will always be some platforms/versions that we miss. Over a period of time, based on experience, inputs from the programming team and historical data from users/analytics/mobile traffic, choosing ‘X’ number of devices becomes easier. The first time is always the hardest, but it is an important step towards optimizing the approach in subsequent test cycles/release plans.

Outsourced Approach



Testing is performed by outsourcing to multiple channels as listed below:
Third party vendors: Testing is outsourced to organizations which claim to have an in house mobile device lab with scores of devices.

Crowdsourcing: Testing is outsourced to organizations that run dedicated mobile test cycles with a well-known crowdtesting community who work in a “Bring Your Own Device (BYOD)” model. This approach also gives a sneak peek into how the crowd (a snapshot of the user base) uses the mobile app and provides input on a large variety of mobile devices under real world conditions.
Mobile Device Lab hosted over cloud: Testing is outsourced to organizations that run cloud based mobile device labs online. Typically, testers will be given access to a web based application to access mobile devices that are connected over a cloud. In this case, gestures, network scenarios, and a few other specific tests cannot be done.

What’s the best way to tame the bull?

In a constant quest to test mobile apps, one of the above approaches can be applied to tame the bull of device fragmentation. In some cases, one or more of the above approaches can be combined together to get better output.

In my experience, a mix of the Shotgun approach and outsourced approach has worked best. I change my strategy based on the context of the mobile app and the expectations of the stakeholders. It has easily taken 2-6 test cycles to determine a good fit for me. The challenge is how we get it right, the first time. It is worth noting that solving the fragmentation problem is not a one-time activity, but a time tested solution that becomes better as we discover newer challenges and deal with them on a case to case basis.

How do you handle mobile device fragmentation challenges on your project? Share your thoughts with me at parimala dot shankaraiah at gmail.


*Image Credits : World Wide Web




22 November, 2014

Managed Crowdtesting - An Augmented Approach to Testing




Testing Industry is going through an ocean of changes. Evolving IT landscape is getting more consumer-driven, open-source and cloud based. With increasing complexity in hardware and software choices, its getting harder and harder to get good testing done on infinite number of platforms, devices, test configurations and a wide variety of user personas.

What is Crowdtesting?

Crowdtesting or Crowdsourced testing is the real-world testing in an on-demand model, delivered through highly skilled & qualified, geographically distributed professionals over a secure private platform.
Crowdtesting solves traditional testing problems as follows:
  • Same brains testing software
  • Fixed/Limited availability of resources
  • Lack of user perspective
  • Lack of fresh ideas


Let me give you a short overview of Managed Crowdtesting which is an extension to how crowdtesting can be done better.

Managed Crowdtesting

A qualified Project Manager, who is typically a proven community leader or a person from the client/the platform company, designs or reviews test strategy, and approves or amends them to cater to client’s specific testing requirements. Each project includes an explanation and access to a forum where bugs and issues are discussed and additional questions can be asked. Testers submit documented bug reports and are rated based on the quality of their reports. The amount the testers earn increases as they report more bugs that are approved by the project manager. The community combines aspects of collaboration and competition, as members work to finding solutions to the stated problem.

Advantages of Crowdtesting

1. Representative scenarios from the real user base
2. Tight feed-back loop with rapid feedback processing and agility
3. Comprehensiveness in use cases, platforms, tools, browsers, testers, etc. that is very hard to replicate in an in-house test lab
4. Cost efficiency
5. Diversity among the pool of testers lends to extensive testing
6. Reduced time to test, time to market and total cost of ownership as most defects can be identified in relatively short time, which leads to significant reductions in maintenance costs

Disadvantages of Crowdtesting

1. Governance efforts around security, exposure and confidentiality when offering a community project to wide user base for testing
2. Project management challenges that stem from the testers’ diverse backgrounds, languages and
experience levels
3. Quality assurance efforts to verify and improve bug reports, identify and eliminate bug duplicates and
false alarms
4. Equity and equality constraints in the reward mechanism with remuneration as a function of the quality of contributions that meets a prescribed minimum standard


Where does Crowdtesting fit best?

Mobile Organizations

With Google, Apple and Microsoft practically giving away their development tools for free, there is a growing developer base creating mobile apps and responsive web sites for android, iOS and Windows platforms. But, it’s easy to underestimate the costs of building and monetizing an app successfully. One way to save costs is to consider crowd testing.
Crowdtesting is most suitable for applications that are user-centric. Users of Mobile and Gaming applications in particular expect the apps to work on thousands of devices from different manufacturers, device sizes, resolutions, network carriers and locations. This calls for not just a group of testers to test on handful of devices and configurations, but for an ocean of users with this kind of diversity.

Growth Stage Startups

With Lean Startup revolution catching up in different parts of the world, startup founders today have gotten smarter by releasing cheaper or free versions of products in beta stage. A few years ago, beta testing happened with a select group that was guarded from the general public. Now, many startups are opening up early versions of applications to users to gather quick and critical feedback. They want to fail faster and learn quicker.
Some applications can be tested in few locations only, some need specific cable connections and network carriers, few others need a specific network connection like 4G LTE or higher, some have needs for specific language users and so on. In such cases, any user might not work. Specific users will be needed in which case an engaged and managed Crowdtesting community comes into play.

Enterprises

Large enterprises can benefit from crowd-sourced testing by simulating a large user base to understand usage patterns and improve on feedback, while ensuring their applications run smoothly on a number of different devices, operating systems, browsers and language versions. Applications with high defect exposure factor post release are good candidates for Crowdtesting.
For e.g. Microsoft released the beta version of its Office 2010 productivity suite, which was downloaded and tested by 9 million people who provided 2 million valuable comments and insights, resulting in substantial product improvements.


Augmenting your test approach

I have heard several businessmen and sales people speak about why offshore testing will work or distributed testing won’t work or how crowdtesting is a magic bullet and so on. With more than a decade of experience in the industry, I can confidently say that there is no magic bullet. Every organization maintains certain ethos with comfortable work culture, talent pool of people and scores of technical debts. Software testing is the least of problems for many organizations, be it 50 years ago, today or even 50 years later. Because, according to customer, software testing consumes money, it doesn’t bring money. In such an overhead situation, selling software testing solutions bundled in different packages to customers as “the most innovative solution of the century” no longer makes sense.
The need of the hour in providing testing solutions is to pitch testing models/solutions as an augmented approach to testing.

Scenario 1 – An organization which employs traditional testing methodologies approaches you for testing

This organization, let’s say, has a mature testing process in place and also has a “Test Center-of- Excellence” for all the testing / QA work that gets done within the organization. How would you add value to them? It’s important to understand the needs of the customer, identify the pain points they are going through as a result of not testing or doing poor testing and pitch a model that fits best for them. Customer might take a couple of test cycles to gauge if that model works well or not.
In this case, if bringing a fresh pair of eyes, then suggesting several new team members in the company/team to initiate testing helps. If it needs to be done on a larger scale, Crowdtesting can be an option. Note that it is not the only option, but one of the options.

Scenario 2 - An organization is looking for diversity in test configurations and devices

A large organization with web or mobile applications accessed from different operating systems, browsers and browser versions, multiple mobile devices, different platforms like Android, iOS, Windows OS, several manufactures, different screen sizes and resolutions. From a cost and time perspective, organizations often find it hard to test on a variety of test configurations. Such a context is suitable for Crowdtesting where a professional testing community works in a Bring Your Own Device (BYOD) model and tests the application, hence giving broader device/platform coverage.

Scenario 3 - An organization wants to solve its regression testing problem

Many legacy applications have a need for regression testing. While new features are conceptualized and implemented, the pain of maintaining existing features from breaking is a big pain. This risk is further aggravated given the number of operating systems, browsers, mobile devices and other test configurations. Regression testing candidates are a great fit for Crowdtesting where the crowd is capable of regressing on a variety of platforms and test configurations within a short period of time.


What does the future hold?

Crowdsourced testing, clearly, has its advantages and limitations. It cannot be considered as a panacea for all testing requirements and the power of the crowd should be diligently employed. The key to great Crowdtesting would be to use it prudently depending on the tactical and strategic needs of the organization that seeks Crowdsourced testing services. It is important for the organization to embrace the correct model, identify the target applications, implement Crowdtesting, explore them for few test cycles; monitor test results and custom make the model to suit their needs.


Why am I talking about Crowdtesting?

I recently joined PASS Technologies, which is into software testing services – Offshore testing and Crowdtesting. While I have been in the Offshore Testing Services for about 11 years now, this is my first experience of how Crowdtesting works. I see a lot of benefits for organizations to adopt Crowdtesting and augment their testing services, be it services based organizations or product based.


What’s in it for a Customer

·         On-Demand Testing
·         Better Test Coverage
·         Faster Test Results
·         Cheaper
·         Scalable Solution


What’s in it for a Tester?

Testers get to “Earn, Learn and Grow” using passbrains platform. Tester benefits include
1. Earning for approved bugs on a per bug payout model
2. Networking with some of the coolest testers who are on our community
3. Recognition as star testers in the community

What are you waiting for? Register on www.passbrains.com as a customer or tester and let us know how passbrains can help you.

References

Content references include thoughts and ideas from Dieter Spiedel, Mithun Sridharan and Mayank Mittal.


07 October, 2014

Interviewed by A1QA



Hello Readers,

I was recently interviewed by A1QA, a Software Quality Assurance company based out of US and UK for their blog. This is the first time, I have been interviewed on technical aspects of my work to great depth. I cover topics like Mobile Apps Testing, User Experience Testing and Crowdtesting for most part.


I am sharing here, just in case, some of you find it useful.

Regards,
Pari

My Interviews

I have been interviewed by many folks. Yet, nothing is in one place so far. This is a placeholder blog post for all my interviews published so far on the World Wide Web.

September 2014


August 2014


March 2014


July 2012

An interview with Parimala Hariprasad : 1 year @ Moolya


June 2011

Parimala Hariprasad - Part 1 @ IT Files
Parimala Hariprasad - Part 2 @ IT Files
Parimala Hariprasad - Part 3 @ IT Files


May 2011

Interview with Parimala Hariprasad @ Testing Circus


December 2010

Interview with Parimala HariprasadSoftware Test and Performance Collaborative 


Regards,
Parimala Hariprasad