btp: Evalutation Criteria

docs/beta_test_plan.md

@@ -141,11 +141,151 @@ With Icing dialer, he can call his collegues and help fix the
 problem, safe from potential Chinese spies.
 
 ## Evaluation Criteria
-- Can a private key be generated
-- Can a normal call be made?
-- Can an encrypted call be made?
-- Can a contact be created / edited / imported / exported?
-- Can a voicemail be listened to?
-- Is the encryption fast enough, light enough to be usable (audible call)
-- Is the encryption strong enough not to be deciphered by a modern (as of 2025)
-  supercomputer?
+### Protocol and lib
+1. Security
+- Encryption Strength: Ensure that the encryption algorithms used (e.g.
+  AES-256, RSA, ECC) are up-to-date and secure.
+- Key Management: Evaluate the mechanism for generating, distributing, and
+  storing encryption keys (e.g., public-key cryptography, Diffie-Hellman, or
+  ECDH).
+- Forward Secrecy: Confirm that the protocol supports forward secrecy, meaning
+  that session keys are discarded after use to prevent future decryption of
+  past communication.
+- End-to-End Encryption Integrity: Verify that no plaintext data is exposed
+  outside the encryption boundary (client-side only).
+- Zero Knowledge: The protocol and library should implement zero-knowledge
+  principles, ensuring that the server cannot access or decrypt the
+  communications.
+- Authentication and Authorization: Evaluate the authentication method (e.g.,
+  multi-factor authentication) and ensure that only authorized users can
+  initiate calls.
+- Replay Protection: Ensure that mechanisms are in place to prevent replay
+  attacks (e.g., using nonces or timestamps).
+
+2. Performance
+- Latency: Measure the round-trip time (RTT) for call setup and audio quality
+  during the call. A good system should aim for low latency to ensure real-time
+  communication.
+- Bandwidth Efficiency: Evaluate the protocol’s ability to minimize bandwidth
+  usage while maintaining acceptable audio quality.
+- Scalability: Ensure that the protocol can handle varying user numbers and
+  call volumes without degradation in performance.
+- Audio Quality (Codec Selection): Assess the choice of audio codecs (e.g.,
+  Opus, G.711) for their impact on call quality at different network
+  conditions.
+
+3. Usability
+- Ease of Integration: Evaluate how easy it is to integrate the library into an
+  Android application, including the availability of well-documented APIs and
+  clear examples.
+- Cross-platform Compatibility: Ensure that the protocol supports multiple
+  platforms (e.g., Android, iOS, desktop) to allow seamless communication
+  between devices.
+- Seamless User Experience: Check for smooth call initiation, handling of
+  dropped calls, and reconnection strategies. The app should handle background
+  operation gracefully.
+- UI/UX Design: Assess the user interface (UI) of the Android dialer for
+  intuitiveness, accessibility, and design consistency.
+- Error Handling and Recovery: Evaluate how the system handles unexpected
+  errors (e.g., network issues, connection drops) and recovers from them.
+
+4. Interoperability
+- Support for Multiple Protocols: Verify if the library and protocol can
+  integrate with existing standards (e.g., SIP, WebRTC) for interoperability
+  with other services.
+- Cross-device Compatibility: Ensure that calls can be initiated and received
+  across different devices, operating systems, and network conditions.
+- Backward Compatibility: Test whether the library is backward compatible with
+  older versions of protocols or legacy systems where applicable.
+
+5. Privacy
+- Data Storage: Evaluate how the system stores any data (e.g., user details,
+  call logs). Ensure that sensitive information is encrypted both in transit
+  and at rest.
+- Data Minimization: Ensure that only the minimum necessary data is collected
+  for the protocol to function.
+- No Call Metadata Storage: Ensure that no metadata (e.g., call logs, duration,
+  timestamps) is stored unless necessary, and, if stored, it should be
+  encrypted.
+
+6. Compliance and Standards
+- Regulatory Compliance: Ensure that the protocol adheres to privacy and
+  security regulations, such as GDPR, HIPAA (if relevant), and other
+  region-specific laws.
+- Open Standards: Verify whether the protocol adheres to recognized open
+  standards for secure voice communications (e.g., ZRTP, DTLS).
+
+7. Reliability
+- Connection Stability: Test the stability of the connection during real-world
+  use cases (e.g., fluctuating network conditions, roaming, mobile data).
+- Error Logging and Monitoring: Assess the logging system to track errors,
+  anomalies, or potential security threats. The system should have proper
+  monitoring to help with diagnosing issues.
+- Redundancy and Failover: Ensure that the system can handle server failures or
+  network issues gracefully with proper redundancy mechanisms in place.
+
+8. Maintainability
+- Code Quality: Review the library for clarity, readability, and
+  maintainability of the code. It should be modular and well-documented.
+- Documentation: Ensure that the protocol and library come with thorough
+  documentation, including how-to guides, API references, and troubleshooting
+  resources.
+- Active Development and Community: Check the active development of the
+  protocol and library (e.g., open-source contributions, GitHub repository
+  activity).
+
+### Dialer
+1. User Interface
+- Design and Layout: Ensure that the dialer interface is simple, intuitive, and
+  easy to navigate. Buttons should be appropriately sized, and layout should
+  prioritize accessibility.
+- Dialer Search and History: Ensure there’s an efficient contact search,
+  history logging, and favorites integration.
+- Visual Feedback: Verify that the app provides visual feedback for actions
+  such as dialling, incoming calls, and call termination.
+
+2. Call Management
+- Call Initiation: Test the ease of initiating a call from contacts, recent
+  call logs, or direct number input.
+- Incoming Call Handling: Verify the visual and audio prompts when receiving
+  calls, including notifications for missed calls.
+- Call Hold/Transfer/Forward: Ensure the dialer supports call hold, transfer,
+  and forwarding features.
+- Audio Controls: Check whether the app allows users to adjust speaker volume,
+  mute, and switch between earpiece/speakerphone.
+- Call Waiting: Verify that call waiting functionality works, allowing users to
+  switch between active calls.
+- Call Recording: If supported, check whether the call recording feature works
+  in compliance with privacy regulations and user consent.
+
+3. Integration with System Features
+- Permissions: Ensure the app requests and manages necessary permissions (e.g.,
+  microphone, camera for video calls).
+- Integration with Contacts: Ensure that the app seamlessly integrates with the
+  Android contacts and syncs correctly with the address book.
+- Bluetooth Support: Test whether the app supports Bluetooth devices such as
+  headsets and car kits.
+- Notifications: Ensure that call notifications work even when the app is in
+  the background or the phone is locked.
+
+4. Battery and Resource Management
+- Battery Usage: Evaluate the dialer app’s impact on battery life during active
+  calls and idle periods.
+- Resource Efficiency: Ensure the app doesn’t excessively consume CPU or memory
+  while operating, especially during idle times.
+
+5. Security and Privacy
+- App Encryption: Ensure that any stored data (e.g., contacts, call logs) is
+  encrypted.
+- Secure Call Handling: Verify that calls are handled securely through the
+  encrypted voice protocol.
+- Minimal Permissions: The app should ask for the least amount of permissions
+  necessary to function (e.g., no unnecessary location or contacts access).
+
+6. Reliability
+- Crash Resistance: Test for the app’s stability, ensuring it doesn't crash or
+  freeze during use.
+- Network Resilience: Test how the dialer handles varying network conditions
+  (e.g., switching between Wi-Fi and mobile data).
+- Reconnect and Retry Mechanisms: Ensure that the dialer can gracefully handle
+  dropped calls and reconnect automatically.