I. Classification
There are two types of testing that take place on mobile devices:
1. Hardware testing:
Equipment includes internal processor, internal hardware, screen size, resolution, memory, camera, radio, bluetooth, wifi, …
2. Software testing or application testing:
- The application works on mobile devices and its functions are tested.
- This is called Mobile App testing to differentiate from earlier methods.
Mobile applications include:
- Native apps_Native apps: An app created exclusively on a platform like a mobile or tablet.
- Mobile web applications_Mobile web apps: Are server-side applications to access websites on mobile using different browsers such as: Chrome, Firefox by connecting to a mobile network or wireless network like WIFI
- Hybrid apps_Hybrid apps: This is a combination of the original application and the web application. They run on devices or offline and they are written using web technologies like HTML5 and CSS.
II. The meaning of mobile application testing
Testing mobile apps is more challenging than testing desktop web apps because:
- Different ranges of mobile applications are accompanied by different screen resolutions and hardware configurations like: hardware keyboard, virtual keyboard and mouse wheel.
- The variety of mobile devices such as HTC, Sam sung, apple, nokia …
- Different of mobile operating systems: Android, window, ios ..
- Differences between OS versions: IOS5.x, 6.x …
- Differences between mobile networks: GSM and CDMA …
- Regular updates: With each update it is recommended to create a new test round to make sure no functionality of the application is affected.
III. The main differences between mobile testing and desktop app testing
- On the desktop, the application is tested on a central processor. For mobile devices, the application is tested on handheld applications: samsung, nokia, apple …
- Mobile app screen resolution is smaller than desktop
- Mobile apps have less memory than desktop computers
- Mobile application uses network connections such as 2G, 3G, 4G, wifi while the desktop uses bandwidth to connect
- The automation tools for the desktop app cannot work on mobile apps.
IV.Mobile application testing types
To address all of the above technical aspects, the following types of testing are performed on mobile applications:
- Usability Testing: Testing usability aspects of mobile applications, to ensure that the mobile application is easy to use and that the user experience is satisfied. row.
- Function Testing: Testing the main functions of mobile apps according to the specification of the device
- Compatibility Testing: Checks your app’s compatibility with native device features to make sure that your app doesn’t interfere with other apps in the device. Test the application on different types of mobile devices, browsers, screen sizes, OS versions – based on the required documentation.
- Interface testing: Check colors, menus, icons, history, settings, navigation on apps, UI consistency across different devices.
- Service testing: Test online and offline application.
- Low-level resource testing: Test memory usage, automatically delete temporary files …
- Performance testing: Test the performance of the application by changing the connection from 2G, 3G to WIFI, document sharing, battery consumption … Check the behavior of mobile apps in Low resources (Memory / Storage Space), the behavior of the mobile website when multiple mobile users visit the mobile app page.
- Operational testing: Test backup and data recovery plan if battery runs out, or data loss while upgrading app from app store.
- Installation tests: Test the application by installing / removing it on the devices.
- Security Testing: Testing an application to verify whether the information system is data confidential.
- Check interruption: Due to the reason mobile devices have much lower memory than desktop, make sure that when there is voice call, SMS, plug in charger, low memory notification while the app is running. running does not cause any conflicts.
V. Mobile application testing strategy
The testing strategy should make sure that all quality and performance match. Some strategies for mobile app testing are as follows:
- Equipment selection: Market analysis and selection of widely used equipment. (This decision is mostly up to the customer. The customer or the app builder considers popular factors or the best major devices on the market needed for the application, which should be used for testing)
- Emulator Environment: Using emulators is extremely useful in the early stages of application development. It allows us to test our application quickly and efficiently. An emulator is a system that runs software from one environment to another without changing the software itself. It includes all the features and works in real environments.
Mobile device emulation environments:
- Device emulator: Provided by equipment manufacturers.
- Browser emulator: Simulates the mobile browser environment.
- OS emulation: Apple provides the emulator for iPhones, Microsoft supplies emulators for window phones, and Google supplies Android phones.
List a few free and easy-to-use emulator app devices
- iPhone Tester: All you need to do is enter the URL in the search field and you will see how it will display on your iPhone.
- Mobile Phone Emulator: Used to test portable devices such as iPhone, Blackberry, HTC, Samsung, …
- MobiReady: With this emulator environment, we can not only test the web application, but also test the code.
- Responsivepx: This environment helps to check the responsiveness of web pages, their display and their functions.
- Screenfly: It is a customizable tool and is used to test websites under different categories.
- After you have achieved satisfaction in developing your emulator application, use your real device for testing.
- Consider cloud computing: Devices run on multiple systems or networks over the internet where applications can be tested, updated and managed. For testing purposes, it creates a mobile website based on an emulator for mobile access and application.
Advantages:
- Backup and restore: The cloud automatically backs up your data. Data storage capacity is unlimited.
- Can be accessed from many different devices anywhere.
- Efficient, easy to use, maintain and update.
- Growing fast.
- It is possible to run the same scripts on several parallel devices.
Defect:
- Poor control: Because of remote controlled applications or 3rd party environments, users are restricted from controlling and accessing some functions.
- Network connection issue: Due to the setting on the internet, the network problem affects the availability and functionality of the application.
- Security matters: Nothing on the Internet is completely secure. Thus, creating more opportunities for hackers.
- Test automatically and test manually
- If the app contains new functionality, test it out manually.
- If the application requires testing once or twice, test it manually.
- Use automatic test for regression test.
- Use automated testing for complex, time-consuming scenarios when doing it manually.
- Network configuration:
- Network configuration is an essential part of mobile testing. Need to test the application on different networks such as 2G, 3G, 4G, wifi …
VI.Testcase for mobile application testing
In addition to basic functional testcases, the mobile application requires the following few special test cases:
- Battery consumption: It is necessary to check the battery consumption while running the application on mobile devices.
- Speed of application: Response time on different devices, with different memory parameters, for different types of networks …
- Data Requirements: Settings to confirm whether a user can download it with a limited amount of data?
- Memory requirements: Download, install and test.
- App Features: Certainly the app doesn’t crash when network crashes or anything.
The test scope depends on some test requirement or the degree of change made to the application .
If the changes are minimal, just do 1 round of testing.
In case of large or complex changes, regression testing should be used in combination. *
VII.How to test mobile apps on Android and iOS platforms
This is very important for Android and iOS testers because they have huge differences: UI differences, app views, coding standards, performance, etc.
Basic difference between testing mobile apps on iOS and Android platforms
- Because there are so many Android devices available in the market and they all have different resolutions and screen sizes. For example, the Samsung S2 size is too small when compared to the Nexus 6. There is a high chance that your app layout and design will be distorted on one of the devices. The probability in iOS is low since there are only countable devices in the market and among many phones with similar resolutions. For example, before iPhone 6 and up, all older versions were the same size.
- The example to confirm the point above is that in Android, developers must use 1x, 2x, 3x, 4x, and 5x images to support image resolution for all devices while iOS only uses 1x. , 2x and 3x. However, it is the tester’s responsibility to ensure that images and other UI elements are displayed correctly on all devices.
- When the market is flooded with Android devices, the code has to be written in such a way that performance remains stable. So chances are your app might work slowly on older version devices.
- Another problem with Android is that a software upgrade isn’t available for all devices. Device manufacturers decide when to upgrade their equipment. It becomes a very difficult task to test everything with both new and old operating systems. Additionally, it becomes a difficult task for developers to modify their code to support both versions. For example, when Android 6.0 came out, there was a big change in the operating system starting to support app permissions. For clarity, the user can also change permissions (location, contacts) at the app level. Now, the test team is responsible for making sure that shows the permission screen on apps that launch on Android 6.0 and up and doesn’t show the license screen on lower versions.
- From a testing perspective, pre-production build testing (i.e. beta) differs on both platforms. In Android, if a user is added to the list of beta users then he can only see the updated beta build on the Play Store if he is logged into the play store with the same email ID added as a user beta.
VIII.Main factors in mobile application testing:
- Define your own test scope
- Everyone has their own testing style. Some testers focus only on what they see from their eyes and others are passionate about everything that works behind the scenes of any mobile app.
- If you are an iOS / Android tester, it is advised that you should at least be familiar with some of the common basic limitations / functions of Android or iOS as it always adds value to your experiment. And of course these are not simple.
- Don’t limit your testing process
- Testing should not be limited to exploring mobile apps and finding bugs. As a QA you should be aware of all the requests sent to the server and the response we receive from it.
- Configure Putty to view history or verify logic for history depending on the content being used in your project. It not only helps you get to know the flow from start to finish of the application, but also helps you become a better tester as you have more ideas and scenarios right now.
Reason :
Nothing comes to this world for no reason. Any statement should have a valid reason behind it. The reason behind the historical analysis is that there are many exceptions recorded in history but they do not show any impact on the UI so we may not notice it.
So should we ignore it? No, we shouldn’t. It doesn’t have any impact on the UI but it could be a future concern. We will likely see our application crash if these exception types continue to grow. As we mentioned application crashing in the final stage, this leads to QA having access to the project’s analytics.
- Cross-platform testing
- Cross-platform interaction testing is crucial. Citing a simple example, let’s say you’re working on a chat app like WhatsApp that supports sending pictures and videos and the app is built on both iOS and Android platforms (Development may or may not sync). Make sure to test the communication capabilities of Android and iOS, the reason is that iOS uses “Objective C”, while Android programming is based on Java and since both are built on different platforms, sometimes it is necessary. Additional application fixes must be made to recognize strings coming from different language platforms.
- Keep track of the dimensions of your Mobile Application
- Another important piece of advice for mobile testers: “Keep checking the size of your app after each release .”
- It should be ensured that the size of the application does not reach the point where the end user wants to download this large application.
- Test the app upgrade
- For mobile app tester, app upgrade scenario is crucial. Make sure your app is not having problems upgrading because the developer team might not have matched the version number.
- Data retention is equally important because in any options the user saved in the previous version should be retained when upgrading the app. For example: The user may have saved his / her bank card details in applications such as Airpay, Momo, …
- The device operating system might not support Sounds Interesting Apps? Yes, many devices may not support your application. Many of you must know that vendors who write applications can all your application SQL queries are not compatible with the device and therefore it will raise an exception and possibly even do not launch the app on that phone. The thing of interest here is: Try using your apps on your own devices except for those you use in the office. It’s entirely possible that you see some problems with your application.
- Checking app permissions Next on the list is Check mobile app permissions. Virtually every second application asks the user to access their phone, camera, gallery, location, etc. Some testers make a mistake by not testing the correct combination of these rights.
- Get an overview of build disavow criteria
- Ultimately, for the most part, you may have encountered a situation where your builds get rejected by Apple. This topic has won the majority of readers’ attention but it’s always good to know Apple’s disclaimer policies. As a tester we get stuck in servicing the technical aspects but still have some elimination criteria that testers might be interested in.
- Always independent
- As a tester, think and work objectively and independently.
- Let’s look at any issues from multiple sides from the Dev Team / Manager.
- If you are passionate about testing, you will be present in all situations. Try to get involved in project activities, before you start testing.
- Keep your apps in the background for a long time (12-24 hours) It sounds weird but there’s a lot of logic behind the scenes that we all don’t understand. Because had the app crashed after launching it, say after about 14 hours from background state. The reason could be anything depending on how the developers coded it.
- Test your application performance
- In the mobile world, the performance of your app impacts how well the app is recognized around the world. As a test group, testing your application’s response becomes too important and more importantly how it works when a large number of users are accessing or using it simultaneously.
The tester’s attitude is very important in the real test environment. When you do it seriously, you will get results. Hopefully, through the above article, you will understand somewhat about testing on mobile applications.
Reference checklist for mobile application testing
The following checklist can help you with some testing of mobile applications:
1. Usability Testing:
- Installation and removal are simple and easy to do.
- The update can be updated easily
- Consistency about: button size, color; font, font size and font color; icons; images on all screens.
- There shouldn’t be too many Pop ups and validation warnings.
- Scroll the page to fit the big data view without the page crashing.
- Simple and concise UI design.
- Allows you to return to the previous page or undo the previous action.
- Tab and Navigation work smoothly.
- Allows users to zoom in / out.
- Display notifications for problems like time out or server not responding.
- Displays error messages for problems such as network loss.
- Charging / disconnecting the charger does not affect the app or appear an inappropriate message.
- Allows to change screen orientation: Portrait- Landscape smoothly.
2.Functional Testing:
- All features must be thoroughly tested.
- All free and paid features must be tested.
- Application must resume at the time when the user stops to make calls or text messages.
- The app must stop operations related to the processes that were abruptly canceled by the device’s settings.
- No data loss occurred, especially applications related to e-retail or banking.
- Do not interrupt calls while the app is running.
- Do not interrupt messages while the app is running.
- Do not interrupt the main device while the app is running.
- Do not interrupt options for social networking sites like sharing, commenting while the app is running.
- Check the different status of the app: close and open, reopen and close, open, close and reopen, …
3.Performance Testing:
- Test the performance and status of the app under conditions such as low battery, small capacity or low network, …
- Test the performance of the app installed on many mobile devices with different operating systems, memory capacity, processing speed, screen size, etc.
- Check the application stability when:
- Multiple users visit at the same time.
- Multiple users install at the same time.
- Perform multi-task at the same time.
- The load is installed just before the overload point to see how the system reacts.
- Time to upload and download multiple types of files.
- Its application and upgrade do not take up too much CPU and memory.
- The operation status of the application when using 2G, 3G and 4G networks.
4.Compatibility Testing:
- Evaluate the application’s compatibility with the environment such as:
- Different operating systems -> Android, IOS, Windows.
- Different browsers -> Firefox, Google Chrome, IE, Safari.
- Application compatibility ratings on different mobile devices have:
- Screen size and resolution.
- The operating system version and memory size.
- The hardware is capable of handling interrupts without crashes.
- Multi-language support.
- Multi-time zone support.
- Check compatibility with connections: Bluetooth, Wi-Fi, USB, etc.
5. Security Testing:
- Does the app store billing or credit card information?
- Does the app require more permissions than it should?
- Does the app use a secure network connection?
- Does the app have a certificate / copyright?
- Does the application ask before accessing the user’s data?
- Is there a maximum number of failed login attempts?
6.Compliance Testing:
- Application UI Test is designed according to instructions of certain operating systems.
- Icons and buttons are used as defined in the operating system manual
7.Accessibility Testing:
- The application is easy to use for the visually impaired.
- The text-to-speech function works correctly.
- Supports high contrast to ensure visibility.
- The application is built according to W3 standards.
References:
https://link.sun-asterisk.vn/sI1Sc3