Showing posts with label Technical. Show all posts
Showing posts with label Technical. Show all posts

Thursday, January 21, 2016

Stream Encrypted Video with Windows Azure Media Services

After a massive break I thought to write on the topic since there are lack of resources. I'm streaming my AES encrypted video with JWT authentication and generating the auth token with Java. If you have Azure Media Service enabled account you are welcome to try this out :).

Following are couple of useful resources in the web for testing this process.

JWT official site and token verification tool - https://jwt.io/
Azure media player - http://amsplayer.azurewebsites.net/azuremediaplayer.html
Base64URL encoder - http://kjur.github.io/jsjws/tool_b64uenc.html
HMAC generator - http://www.freeformatter.com/hmac-generator.html

At first we need to have encrypted videos uploaded to our Azure account. Following are the steps to doing that.
1. Login to manage.windowsazure.com
2. Go inside your media service account
3. Click on Upload a video file link under Managemant Tasks section (or else click on Content tab and then click on Upload Content link) and upload a video file.
4. Click on Process icon at the bottom, and click OK with default values.
5. Click on Sync Metadata icon at the bottom (Probably this will take some time)
6. Click on Encryption > Enable AES Encryption (If the Encryption icon not yet enable, logout and login again and try. It may take some time to enable that)
7. Click on Publish icon (SS for 4-7 https://www.diigo.com/item/image/5j6wi/wdb1)

Now you can copy the encrypted video publish URL which will be like : http://chanaka.streaming.mediaservices.windows.net/481ea237-7f1f-48b8-a4c1-096f34c10cd6/Best%20Sweet%20Minions%20Video%20-%20Let%20The%20UFO%20Go%20Home.ism/Manifest

Then go to Content Protection tab and set up the necessary attributes like this SS and click on manage verification keys button and regenerate primary key. And don't forget to save :)

For streaming the above encrypted video, we need to create a JWT auth token which is the tricky part. Following is sample java code to generate the token.


package azurejwttoken;

import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import org.apache.commons.codec.binary.Base64;

/**
 *
 * @author chanaka
 */
public class AzureJWTToken
{

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args)
    {
        // Use the Primary Verification Key which you regenerated
        String          secret              = "YEAIfGy6cIXy1xhcIdgEoBfiWNoI3zq2FCKVQhdZXMcHGN+BAwicafgbRscsVZj1Us3RlWihphY1XMBuQOhlBg==";
        
        long            validitymillis      = 15 * 60 * 1000; // 15 minutes
        long            nowMillis           = System.currentTimeMillis();
        long            expireSeconds       = (nowMillis + validitymillis) / 1000; // Use seconds instead milli seconds
        
        String          jsonHeaderString    = "{\"typ\":\"JWT\",\"alg\":\"HS256\"}";
        
        // Use your ISSUER and SCOPE attributes properly here
        String          jsonBodyString      = "{\"iss\":\"http://mytest.com/\",\"aud\":\"urn:jwttest\",\"exp\":" + expireSeconds + "}";
        
        String          token               = getAuthToken(secret, jsonHeaderString, jsonBodyString);
        
        System.out.println("Token : " + token);
    }
    
    public static String getAuthToken(String secret, String jsonHeaderString, String jsonBodyString)
    {
        String          token       = null;
        
        try
        {
            Mac             sha256_HMAC         = Mac.getInstance("HmacSHA256");
            SecretKeySpec   secret_key          = new SecretKeySpec(Base64.decodeBase64(secret), "HmacSHA256");
            
            sha256_HMAC.init(secret_key);
            
            String          content             = Base64.encodeBase64URLSafeString(jsonHeaderString.getBytes()) + "." + Base64.encodeBase64URLSafeString(jsonBodyString.getBytes());
            String          hash                = Base64.encodeBase64URLSafeString(sha256_HMAC.doFinal(content.getBytes()));
            
            token           = "Bearer%3D" + content + "." + hash;
        }
        catch (NoSuchAlgorithmException ex)
        {
            Logger.getLogger(AzureJWTToken.class.getName()).log(Level.SEVERE, "Error occured while generating AuthToken : NoSuchAlgorithmException", ex);
        }
        catch (InvalidKeyException ex)
        {
            Logger.getLogger(AzureJWTToken.class.getName()).log(Level.SEVERE, "Error occured while generating AuthToken : InvalidKeyException", ex);
        }
        catch (Exception ex)
        {
            Logger.getLogger(AzureJWTToken.class.getName()).log(Level.SEVERE, "Error occured while generating AuthToken", ex);
        }
        
        return token;
    }
    
    /**
     * In case you need to convert base64 string to URL safe manually ()
     * 
     * @param base64EncodedString
     * @return 
     */
    public static String replaceUrlSafeChars(String base64EncodedString)
    {
        base64EncodedString = base64EncodedString.replace('+', '-');
        base64EncodedString = base64EncodedString.replace('/', '_');
        base64EncodedString = base64EncodedString.replace("=", "");
        
        return base64EncodedString;
    }
    
}

Using the JWT site we can verify the generated auth token. Just paste your token to JWT Debugger without the "Bearer%3D" part like this SS.

If it's correct you should be able to play it with Azure media player like this SS. Note that token will expire after 15 minutes according to our code. You can increase that accordingly.

Finally you can stream your video with following hard coded HTML source. Make sure you open the HTML with web server.

<!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    </head>
    <body>
  <iframe src="//aka.ms/azuremediaplayeriframe?url=http%3A%2F%2Fchanaka.streaming.mediaservices.windows.net%2F481ea237-7f1f-48b8-a4c1-096f34c10cd6%2FBest%2520Sweet%2520Minions%2520Video%2520-%2520Let%2520The%2520UFO%2520Go%2520Home.ism%2FManifest&protection=aes&token=Bearer%3DeyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwOi8vbXl0ZXN0LmNvbS8iLCJhdWQiOiJ1cm46and0dGVzdCIsImV4cCI6MTQ1MzM1MjE2MH0.j6pe2Ej9aJJy0AvaqR7OI9FCcNTS1qI8y_EGzNYKT4g&autoplay=false" name="azuremediaplayer" scrolling="no" frameborder="no" align="center" height="280px" width="500px" allowfullscreen></iframe>    </body>
</html><!--
Iframe src dynamic url parameter :
    url : URL encoded encrypted video URL - java.net.URLEncoder.encode(encryptedVedioURL, "UTF-8")
    token : The token generated from the java code
--> 

Hope you enjoy. Happy coding... :)

I like to thank Saliya Randunu for providing me lots of resources :)

Wednesday, April 9, 2014

Getting Started with WSO2 ESB Connectors

Enterprise Service Bus (ESB) is the infrastructure which provides the implementation of the concept of Service Oriented Architecture (SOA). The connectors are the new concept coming with the wso2 ESB version 4.8.0 and above. Connector is an intermediate API layer, which will represent a real world developer API, while being in the ESB itself. With this post, the initial steps to create a connector will be described.

Download Maven Dependencies:
You need to checkout the source code from here and run maven install command on it. It will download set of required dependencies to your local maven repository. Following are the commands if you have a command line SVN and maven client.

$svn co https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/mediation/mediation-library/connector-template-utils/org.wso2.carbon.mediation.library.connectors.connector-archetype/
$cd org.wso2.carbon.connector/
$mvn clean install

Generate Connector Folder Structure:
Run the following maven command to generate maven project sample connector code.

$mvn archetype:generate -DarchetypeGroupId=org.wso2.carbon -DarchetypeArtifactId=org.wso2.carbon.mediation.library.connectors.connector-archetype -DarchetypeVersion=4.2.0 -DgroupId=org.wso2.carbon.connector -DartifactId=org.wso2.carbon.connector -Dversion=4.2.0


You may need give a name for the connector while processing the above process. (Let's proceed with entering googlebooks as connector name.) If the project creation success, you are ready to develop a connector with the following directory structure.

Fig. 1 : Connector Directory Structure

Before developing, there are some main components which you need to know.

Synapse Template : This is the component which calling to actual end point with the given request data. This will be placed in resources directory.

Proxy : This will handle the request coming inside to the ESB and response coming from the actual end point. This is not a part of connector, but need to invoke the connector.

Request : The request which follows the API published from the connector. This request can be REST or SOAP.

Connector for which API ?
There are large number of services which is provide in the world wide web to do a specific functions. In order to use these for developers, there will be one or several web service APIs  like REST, SOAP or a client libraries in one or several languages are provided by the service provider. Out of these, REST will be the simplest one that others since it has less work. Here we are using Google Book REST API which does not need authentication for some of services like volume search.

First we'll go the Google Books REST API documentation and identify the categorization of the services in GoogleBooks API. The services are categories like Bookshelf, Volume, Bookshelf.volumes etc. So we'll create new directory "volume" under resources directory, to place services in the Volume category.

Writing Synapse Template :
You can have a look on the sample Synapse template generated at helloWorld directory. Following synapse template is providing the service at list volume. This should be placed under "volume" directory as listVolume.xml.

<template name="listVolume" xmlns="http://ws.apache.org/ns/synapse">
    <parameter name="searchQuery" description="Full-text search query string." />
    <sequence>
      <property name="uri.var.searchQuery" expression="$func:searchQuery" />
        <call>
            <endpoint>
                <http method="get"
uri-template="https://www.googleapis.com/books/v1/volumes?q={uri.var.searchQuery}" />
            </endpoint>
        </call>
    </sequence>
</template>

Create a component.xml file in "volume" directory, with following content.

<?xml version="1.0" encoding="UTF-8"?>
<component name="googlebooks_volume" type="synapse/template">
    <subComponents>
        <component name="listVolume">
            <file>listVolume.xml</file>
            <description>List volumes matching for the given query.</description>
        </component>
    </subComponents>
</component>

Modify the conector.xml file by adding new dependency as below,

<?xml version="1.0" encoding="UTF-8"?>
<connector>
    <component name="googlebooks" package="org.wso2.carbon.connector" >
         <dependency component="helloWorld"/>
         <dependency component="googlebooks_volume"/>
         <description>wso2 sample connector library</description>
     </component>
</connector>

Now you can build the connector code and test the implemented service foe volume search method. To build the connector, run maven clean install command from the org.wso2.carbon.connector directory.

$mvn clean install

This will generate googlebooks.zip file in the target directory. You can upload this zip file through the ESB interface under connector category. Make sure you enabled the connector after uploading.

Writing Proxy :
Now you can invoke the implemented service for search volumes, by writing a proxy through ESB server interface. (Home>Manage>Services>Add>Proxy Service>Custom Proxy>switch to source view)

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="googlebooks_listVolume"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
         <property name="searchQuery" expression="json-eval($.searchQuery)"/>
         <googlebooks.listVolume>
            <searchQuery>{$ctx:searchQuery}</searchQuery>
         </googlebooks.listVolume>
         <respond/>
      </inSequence>
      <outSequence>
         <log/>
         <send/>
      </outSequence>
   </target>
   <description/>
</proxy>


Request :
Using a REST client, send following request, (you can use POSTMAN)

POST /services/googlebooks_listVolume HTTP/1.1
{
    "searchQuery":"aladin"
}

Hope you get the correct response ;)

Thursday, November 21, 2013

OAuth2 with LinkedIn

OAuth2 basically allows third party application, to access protected resource of a specific user. You may have seen some web applications/sites allows you to login through your Google, Facebook, LinkedIn etc accounts. As an example just try this with any mentioned account (Google, Facebook, Yahoo etc.). After giving your username password credentials you will be notified the authorities which the third party application will get if you agreed to proceed.

Some important term definitions in OAuth2:
  • Resource Owner - The person/entity which capable of granting the access to a protected resource.
  • Resource Server - The server which contains the protected resources, and capable of deal with access tokens in protected resource requests.
  • Client - The application which requests for protected resources on behalf of resource owner.
  • Authorization Server - The server which authorize resource owner, and issue access tokens to the client

Following figure shows basic flow of OAuth2 authentication process.




If you are really interested in digging more details, OAuth2 RFC will be the best resource. I strongly recommend you to go through at least up to 1.6 section in the RFC.

With this blog post, I show you how third party java application can access protected LinkedIn resources.

1. Register an application on LinkedIn. - This basically helps to create a link between LinkedIn and your application (third party application).

2. Get an access token - The access token represent the credentials to access protected resources. So your application does not need to store resource owner's credentials, but the access token.

3. Make API calls - Yes, after obtaining access token you are free to call API methods with that, until it expires.

LinkedIn official documentation contains detailed description on the above steps. You can follow those and familiar with this matter. Make sure you tried both Authorization code request and Access token request.

Now we need our application to store access token. For this we have to do some coding in the application. We can easily get authorization code with a GET request. Then we need a POST request to get access token using the authorization code and other required parameters.

So, we can get the authorization code from request parameters in the redirect uri, and make server side code request to get the access token. For this we have to use a HTTP client library. You can use Apache HTTP client library components to approach this. For process the responce you can use XML parser like DOM or SAX. Try out the sample application yourself after reaing README file. Happy coding !


Thanks to mkyong for making me comfortable with his SpingMVC example.

Tuesday, January 8, 2013

Limit textarea character count in IE8

   After long time I'm going to start blogging again. Yes, it's on your favorite browser IE :). As you know, with HTML 5 there is lots of additional features support for major browsers except IE. However it's nice to hear that IE10 is having more support for HTML 5.

     With the HTML 5, maxlength attribute was introduced for textarea HTML element. But IE 8 does not have support for this. I did some work around to find a simple solution for this and seems I found it.
 
 


    Make sure you are using HTML 5 doctype. Note that I have target only IE here, but not the already support browser for maxlength attribute in teaxtarea elements.

<!DOCTYPE HTML>
<html>
  <head>
    <script type="text/javascript">
      function getCaret(el) {
        if (el.selectionStart) {
          return el.selectionStart;
        } else if (document.selection) {
          el.focus();
          var r = document.selection.createRange();
    
          if (r == null) {
            return 0;
          }
    
          var re = el.createTextRange(),
          rc = re.duplicate();
          re.moveToBookmark(r.getBookmark());
          rc.setEndPoint('EndToStart', re);
          return rc.text.length;
        }
        return 0;
      }
 
      function limitPasteData(obj, limit) {
        // Do it with Internet Explorer
        if (window.clipboardData) {
          if (document.selection && document.selection.createRange) {
            obj.focus();
            var modifiedTxtLen;
            // If text has been selected
            if (document.selection.createRange().text.length > 0) {
              modifiedTxtLen = obj.value.length - document.selection.createRange().text.length + window.clipboardData.getData("Text").length;
    
              if (modifiedTxtLen > limit) {
                var neededLen = limit - (obj.value.length - document.selection.createRange().text.length);
                document.selection.createRange().text = window.clipboardData.getData("Text").substring(0, neededLen);
                return false;
              }
            } else {
              modifiedTxtLen = obj.value.length + window.clipboardData.getData("Text").length;
              if (modifiedTxtLen > limit) {
                var neededLen = limit - obj.value.length;
                var neededTxt = window.clipboardData.getData("Text").substring(0, neededLen);
                var mofifiedText = obj.value.substring(0, getCaret(obj)) + neededTxt + obj.value.substring(getCaret(obj), obj.value.length);
                obj.value = mofifiedText;
                return false;
              }
            }
            // If not exceed limit just paste it
            return true;
          }
        }
      }
    
      function limitTypingData(obj, limit) {
        // Other browsers may have support for maxlength attribute
        if (navigator.appName == 'Microsoft Internet Explorer') {
          if (obj.value.length > (limit - 1)) {
   
            if (document.selection && document.selection.createRange) {
              obj.focus();
              // If only text has been selected, allow to type once
              if (document.selection.createRange().text.length > 0) {
                return true;
              }
            }
            return false;
          }
        }
      }
    </script>
  </head>
  <body>
    <textarea id="txtArea" cols="20" rows="5" maxlength="15"
        onpaste="return limitPasteData(this, 15);"
        onkeypress="return limitTypingData(this, 15);" ></textarea>
  </body>
</html>

       But remember that the behavior of these functions in IE8 will not 100% equal to default maxlength behavior of supporting browsers. But we can still do the work with this.

Apart from that I would like to notice you that, new line character(s) may be changed in IE than others. IE always take it as '\n\r' and other browsers may take it as '\n'. So that, either you have to handle that or use additional space from the back end database fields.

Have a fun with IE 8 :D

Thursday, September 6, 2012

Code to interfaces, not to implementations

      In object oriented programming, interfaces are commonly used to implement more flexible software. Interfaces are very useful to extend and improve  a software application. While designing a software its a good practice to use interfaces where applicable.

           Lets think about a simple CricketPlayer class (Figure 1) with a method play(). There can be several classes and methods using CricketPlayer class. Think we need to give the support for Football player as well within this software. Then we need to repeat the wheel for new purpose(Figure 2).

Figure 1 : Initial cricket class

Figure 2 :Modification for support football player

        But just think if we use common interface (Figure 3) for any type of player, we can easily get rid of the problem. Player interface will have play() method which may be different behaviour for different players and getSport() method which represent the type of the player (sport). CricketPlayer and FootballPlayer subclasses can implement their own behaviours according to related declarations in the Player interface. Coding to this interface will save us with very limited changes, while any kind of player class is going to use in our software. Using an interface software can be extend easily and will be more generic.

Figure 3 : Player interface

        Recently I saw practically very useful place where interfaces are using. Most of the industry applications are using the MVC architecture to better maintainability of the code. There may have requirements to use the same applications with different DBMS for different clients. Using an interface with n-tier architecture, this support can be easily implemented.

          Following diagram (Figure 4) shows the way of using this stratergy in n-tier architectural application. From the Controller classes which deals with request and responses while dealing with the business logic of the application, calls to the service classes to do specific data manipulation, transaction etc. Service classes calls to Dao classes which deals with data transactions. Service class doesn't directly use either MySqlUserDoa which has MySQL support or XQueryUserDoa which has XQuery support. It only care about BaseUserDao interface, and doesn't care about real object underling the BaseUserDao interface.

Figure 4 : Dao interface

      Implementaion under this process is quite simple. Control class read configuration file (or from any other way) and get the Query Language supposed to use according to the requierments. Then create an object of supported Dao class and assigned to a variable of BaseUserDao (interface) type. After this initializing process of BaseUserDao type variable, it can be used by the service classes inside their functions.

BaseUserDao userDao;
if (queryLanguage == 'MySQL') {
    userDao = new MySqlUserDao();
} else if (queryLanguage == 'XQuery') {
    userDao = new XQueryUserDao();
}

        Since application is not creating concrete classes (objects are dynamically created in run time and will not directly creating objects for any case), it will not be limited. And since it uses an interface, switching for a new DBMS can be achieved by only adding new file which inherit the interface and modify the object creating process(modify above code) and giving the new DBMS as parameter, using a configuration file or any other way.

      Coding to interfaces, not to implementations principle is a good practice for a good software and it make our work more easy. If you are thinkng of creating extendable software application, this practice will very helpful. So, its time to play with interfaces. :)

Tuesday, August 28, 2012

Singleton with PHP

        Singleton pattern is under creational design patterns which deals with mechanisms of creating objects according to the situation. I recently used this pattern with PHP. It's very important to get to know use of this pattern.

              Singleton pattern can be used for always keep one object of a specific class while an application is running. It save the memory which is going to allocate for new objects of the class in every where inside the application. Singleton practically used for logger classes which provide global logging access point whithout logging each and every application after the main loggin operation, configuration classes which consist configuration data of an application, access resources which are in shared mode like multi threading applications, Factory design pattern which dynamically create the objects etc.

         It's very easy to implement singleton with PHP (as well as other languages). What we need to do is make a private constructor and control the creating object of this class by a public method. Following code snippet shows how to do this.

class MySingleton {

    private static $instance = null;

    public static function getInstance() {
        if(is_null($this->instance)) {
            $this->instance = new MySingleton($params);
        }
        return $this->instance;
    }

    private function __construct($params) {
        // set parameters
    }

    public function doSomething() {
        // some code
    }

}

         Since the constructor is private only the class itself can be access the constructor. getInstance method is accessible for anywhere and since that is a static method it can be used anywhere without creating an object of this class. This function handles the situations of the need of a new instance of MySingleton class by calling the private constructor if needed. Static variable is used because of the restriction of using non-static fields inside static functions.

In the places where needs to call doSomething method in MySingleton class, we call as,
MySingleton::getInstance()->doSomething()
or, if the functions inside the class is frequently used,
$my_singleton = MySingleton::getInstance();
$my_singleton ->doSomething()

It's very simple. Hope this helpful. Have a fun with singleton. :)

Sunday, April 15, 2012

Test Driven Development with PHPUnit

       TDD (Test Driven Development) concept is common practice of most of the companies which involve with the long time projects. It helps to keep the static behavior of functions over the development period. The functions are implemented after implementing test methods (unit tests) for that function. After implementing the function, run the tests and if any test is failing, again modify the function. This process repeats until the all tests are passing.

                       Test method is a function which check whether the specific behavior of a function is correct. Any function can have at least two test methods. one positive test and one negative test. Most probably a function may have many test methods. The class in which all the test methods are placed is called Test Case. By setting up a test suite for this class, the all tests methods can be run at once.

         In long time projects like open source projects, the functions are frequently modified by the developers during a long time. If the modified function is not given the previous behavior of that function, the output for other places will be going wrong. It can be cause to failures. By writing tests this problem can be reduced. Before and after modifying the test cases, by running the tests, developer can be verify the function behave correctly. As well by going through the test cases helps even understand what actually function does.

      To follow this TDD concept with Java, JUnit framework can be used. As well with PHP, PHPUnit framework can be used. Let's do some experiments with PHPUnit with Ubuntu OS.

Install PHPUnit framework
$sudo apt-get install phpunit

Now create your test case as a sub class of 'PHPUnit_Framework_TestCase' having just one dummy method to verify we can forward without issue.

<?php

    require_once 'PHPUnit/Framework.php'; // installed file with phpunit framework which is located at : /usr/share/php/PHPUnit/

    class MyClassTest extends PHPUnit_Framework_TestCase {

        // This function just return true
        function testCheckAllWorksFine() {
            $this->assertTrue(true);
        }

    }

?>

Run this MyClassTest.php file as :
$phpunit MyClassTest.php

If the result is as following (Fig. 1), you can proceed the work with phpunit.
Fig. 1


Characters it returns :

.     : pass
F    : fail
E    : error


The functions starting from 'assert' is implemented in the framework. As well there are more useful functions we can use with this framework. You can find more assert functions by visiting http://www.phpunit.de/manual/3.6/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.assertions.

I have used following useful assert functions for this article :
assertTrue() : Check whether the parameter is True
assertFalse() : Check whether the parameter is False
assertEqual() : Match two results



Start work

1. Requirement : need a function which accept two strings and return the length of the most lengthy string, if both arguments are not strings return false

* First just declare your function properly with arguments
<?php
    class MyClass {   

        function getMaxLength ($str1, $str2) {    

        }

    }
?>

* Now start to write test methods,
a). Check the arguments
<?php

    require_once 'MyClass.php';
    require_once 'PHPUnit/Framework.php';

    class MyClassTest extends PHPUnit_Framework_TestCase {

        var $obj;

        // This function calls before any test merhod run
        function setUp() {
            $this->obj = new MyClass();
        }

        // Positive : Test getMaxLength for correct arguments
        function testGetMaxLengthForAtLeastOneIsString() {
            $paramArray = array("str1", 2);
            $result = $this->obj->getMaxLength($paramArray[0], $paramArray[1]);
            $this->assertEquals(4, $result);
        }

        // Negative : Test getMaxLength for non of arguments are string
        function testGetMaxLengthForNoStrings() {
            $paramArray = array(array(1,2), 12.3);
            $result = $this->obj->getMaxLength($paramArray[0], $paramArray[1]);
            $this->assertFalse($result);
        }

    }

?>


The setUp() will call automatically before running any test method. It can be used for create a fresh objects, load data for databases if using etc. All test methods must be start with 'test'. Using the function name which is going to use by specific test method regardless of the length of the test method name, will be increase the readability and understandability while reviewing later.


b). Check whether the function returns an integer
// Test getMaxLength for output integer for correct arguments
function testGetMaxLengthForCorrectReturnType() {
    $paramArray = array("str1", 2);
    $result = $this->obj->getMaxLength($paramArray[0], $paramArray[1]);
    $this->assertTrue(is_int($result));
}



c). Check whether is returns the length of the most lengthy string
// Test getMaxLength for Length of the lengthy string
function testGetMaxLengthForCorrectLength() {
    $paramArray = array("short_string", "lenghty_string");
    $result = $this->obj->getMaxLength($paramArray[0], $paramArray[1]);
    $this->assertEquals(14, $result);
}


Now the most needed test cases are implemented. From the above test methods, some scenarios like check the function return length of the string for arguments having only one string, is already covered. Now just run the test and see most of them are failing. Its time to start implement the actual function in a way that all te test cases are passing.


Function can be written as :
function getMaxLength($str1, $str2) {   

    if(is_string($str1) || is_string($str2)) {
        if(is_string($str1) && is_string($str2)) {
            $max = (strlen($str1) >= strlen($str2)) ? strlen($str1) : strlen($str2);
        } elseif (is_string($str1)) {
            $max = strlen($str1);
        } else {
            $max = strlen($str2);
        }
        return $max;
    }
    return false;

}


Now again run the test. See whether all the test are passing or not. If not your function may have some issues. Improve the function and again run the test. Do this process until all the test are passing.

The example we discussed was very easy one. But in real applications we have use more complex algorithms and tactics dealing with databases. To check the functionality of the data transaction functions, we can use a test database with some preloaded data. Inside the setUp() function we can create tables and load the data to the test database. As well using the function tearDown() which automatically call after execution of any test method, we can clean the data from the database.

Sometimes applications with MVC architecture have two or more separate layers in Data Layer. In such kind of situation, functions need to test independent from their layers. Lets assume in our application there are two layers in Data Layer. One is data transaction layer which directly deal with the database and other one is data processing layer which process data before or after deal with database through data transaction layer. So, while writing test for a function in data process layer which get some data by calling a function in data transaction layer, we should verify that the correct data set is returned from the function in data transaction layer. To do this we can use Moc Objects.

Lets go through one example.
* getStrings() method in DataTransaction class return an array with two strings (No need of even declare this class)

* getProcessedResult() method in DataProcess class, get the array of two strings by calling the above mentioned function (getStrings()), and return the length of the most lengthy string out of them. This class contain getter and setter methods for use DataTransaction class.
(Declare the getProcessedResult() function and implement getter and setter methods)

<?php

    class DataProcess {

        private $dataTransaction;

        function getDataTransaction() {
            if(is_null($this->dataTransaction)) {
                return new DataTransaction();
            }
            return $this->dataTransaction;
        }

        function setDataTransaction($dataTransaction) {
            $this->dataTransaction = $dataTransaction;
        }
  
        function getProcessedResult() {
    
        }

    }

?>

* Now write the test class using the Moc Object of DataTransaction class

<?php

    require_once 'DataProcess.php';
    require_once 'PHPUnit/Framework.php';

    class TestMoc extends PHPUnit_Framework_TestCase {

     var $dataProcess;

     // This function calls before any test merhod run
     function setUp() {
                $this->dataProcess = new DataProcess();
     }

     public function testGetProcessedResult() {

         // Create a Mock Object for the DataTransaction class
         // mocking only the getStrings() method.
         $dataTransaction = $this->getMock('DataTransaction', array('getStrings'));
  
         // Set up the expectation for the getStrings() method
         // to be called only once and with no arguments
         // and return value is array("abc", "abcde")
         $dataTransaction->expects($this->once())
                         ->method('getStrings')
                         ->with()
                         ->will($this->returnValue(array("abc", "abcde")));
  
         // Set the mocked DataTransaction object to 
            // DataProcess object using setter
         $this->dataProcess->setDataTransaction($dataTransaction);
  
         // Call the getProcessedResult() method on the $dataProcess object
            // which we expect to call the mocked DataTransaction object's
         // getStrings() method with no arguments.
         $this->assertEquals(5, $this->dataProcess->getProcessedResult());

     }

    }

?>

Here, even we didn't implement (even declare) the DataTransaction class, we could run the test using Moc Object. So that we can test functions independent from the layer it contains and other needed functions.


Setting up test suite

Test suit can be used to mange which test cases should run. We can run even all the test methods in several test cases at once. Test suite can be create as following example code snippet.

<?php

    require_once 'PHPUnit/Framework.php';

    class AllTests {

        public static function suite() {

            $suite  = new PHPUnit_Framework_TestSuite("PHPUnit");
            // Add needed test cases
            $suite->addTestFile("MyTestClass1.php");
            $suite->addTestFile("MyTestClass2.php");

            return $suite;

        }

    }

?>


With PHPUnit framework the percentage of the functions which covers from the unit tests can be obtained as a coverage report. Following command can be used to obtain a coverage report. (probably you need to install xDebug : http://xdebug.org/docs/install)

$phpunit --coverage-html ./report AllTests.php

From this report we can find out the uncovered code segments. Some of code segments in MyClass.php has not covered from the unit tests. To cover them again test can be implemented.

So, this is a basic idea of how to follow test driven development with PHPUnit framework. Hope this helpful to you.

Saturday, December 24, 2011

Use of Subversion client with sourceforge

             Subversion (SVN) is Centralized, open-source Version Control System. SVN  manages files and directories and changes made to them over a period of time, while supporting to shared development with features like rollback code conflict resolve. As well this allows to recover older versions of the software (data) or examine the history of how the code-base changed. SVN is developed in order to overcome the bugs and misfeatures of CVS.

                 In SVN there are two ends to deal with. One is the Subversion Repository which contains all the managed files and directories, and the other one is commandline or GUI client application to manage files from the client's end. It is important to know the concept of SVN repository before starting work with SVN.

           Repository is a file server which is not behave like an usual one. It has it's own revisioning system. After any authorized person change a file in repository, the state of the code-base will change and it is given a index to identify that state which is new code-base. This index is called revision number. At the initial state of the repository revision number is '0'. For any change in repository the revision number will increase by one (Figure 1). So that people can refer the older versions of the code-base.

Figure 1

Usually repository contain three main directories containing the code-base.
  • trunk   - code-base of main development process
  • branch - copy of specific revision, to implement new features(which takes some time to finish) concurrently and independently on top of that (Figure 2)
  • tag      - some snapshots of the trunk. Can store stable versions of the code-base

Figure 2

Even in branching process, the revision numbers are depend on changes of the branches (Figure 3). How ever at the end of the branching process of a particular branch, containing code should be merge with the main code-base (in trunk).

Figure 3

     Now I think you have an idea on how the repository behave. To access SVN repository there are several methods. Following table shows the URL schemes map to each access method.


SchemeAccess Method
file:///Direct repository access (on local disk)
http://Access via WebDAV protocol to Subversionaware Apache server
https://Same as http://, but with SSL encryption
svn://Access via custom protocol to an svnserve server
svn+ssh://Same as svn://, but through an SSH tunnel

              As I mentioned earlier, the second end of SVN is the application, in client's end. I'll explain how we can generally use SVN client applications with Linux & windows environment.


Setup a repository

                  We can get use of sourceforge in order to setup a repository. sourceforge is free repository provider for open-source projects. What you have to do is register for the sourceforge and create a project. Make sure to select the version control system as SVN (default is git) while you are creating the project. sourceforge itself can add folders and files. But since we are in another track forget it for a moment. Under the code tab of your project home page, you can see the command you should use to get a checkout for linux based system. (make sure you have login)


With UBUNTU-10.04 OS

         SVN commandline client can be used with Linux based systems. After installing the SVN commandline client, files can managed via executing the commands in command prompt.

install SVNcommandline client on your machine
$ sudo apt-get install subversion libapache2-svn
first go inside that directory and add source files to the repository.
$ cd /home/source_folder
$ svn import --username=your_username -m "Meaningfull_message_about_the_source_changes_for_users" url_of_repository
note : the repository url you can obtain from the sourceforge (sourceforge -> your project -> code tab)

In sourceforge project your initial file upload will appear as revision number 2. This is due to sourceforge itself create the general folder structure of repository and it will appear as revision number 1.


To start developing with SVN, first create a new folder and then get a checkout from the repository (copy of the latest code-base in repository)
$ sudo mkdir /var/www/myproject
$ cd /var/www
$ svn co --username your_username url_of_repository myproject
Now you can start further implementation on that checkout. If you modify an existing file it will already prepared to commit (upload) to the repository. But if you add a new file or a directory first you have to make prepare it to commit.
$ svn add new_file_1 new_file_2
$ svn ci -m "Meaningful_Message" new_file_1 existing_file_1 existing_file_2
If you want to commit all the files you changed, you use above code without the file paths. A summery of some useful SVN commands are shown below.

information on usage of SVN
$ svn --help
initial commit
$ svn import --username=your_username -m "Message" URL
get a checkout
$ svn co --username your_username URL project_folder
make prepare to commit the newly added files (unversioned files)
$ svn add file_path
see the states of the files (whether versioned or unversioned)
$ svn st
note: The meaning of the symbols shown for statuses,
? - unversioned file
M - locally modified file
A - schedule to add the file
D - schedule to delete the file
G - merged file
C - conflicted file while updating (file is waiting for user resolve the conflict)
! - file is missing or incomplete (removed by non-svn command)
~ - versioned file obstructed by some file of a different kind
I - file ignored

get latest code to working copy without changing local modifications
$ svn up
note: If any file is about to conflict (overwrite) commandline client itself give some options to user to agree,
p - mark the conflict to be resolved later (postpone)
df - show all changes made to merged file (diff-full)
e - change merged file in an editor (edit)
r - accept merged version of file (resolved)
mf - accept my version of entire file and ignore their changes (mine-full)
tf - accept their version of entire file and lose my changes (theirs-full)
l - launch external tool to resolve conflict (launch)

h - show this list (help)

resolve conflicted file before commit (after postpone conflict we can manually resolve it)
$ svn resolved conflicted_file_path
commit all local changes
$ svn ci -m "Message"
note: to commit specific files, add file paths at the end of the above command separating each by a space

remove all the local changes of a working copy (become to base revision)
$ svn revert -R .
note: to revert specific file replace '-R .' with file path

information on base revision (revision of working copy on top)
$ svn info
modifications between working copy and base revision
$ svn diff
get file differences to a file
$ svn diff > ~/Desktop/path/filename.diff
apply patch (diff file) to working copy
$ svn patch -p0 ~/Desktop/path/filename.diff

With Widows 7 OS

             SVN GUI client can be used for windows based systems. TortoiseSVN is the GUI client of SVN. After Installing Tortoise SVN on a machine, files can be managed in UI level. To start the development, go to any directory, under which this project should be created, then right click and select 'SVN checkout' option. Then you can give the repository URL and the local project directory name. If you have already created a project in sourceforge, from the SVN command appeared under 'code' tab, you can fetch out the repository URL.


After any improvement in the code, the corresponding file and all it's parent directories are shown as modified by a red mark on the file or directory. Using the right click options, the modified code can be committed (SVN commit) even while neglecting some of the modified files by selecting them. As well you can see lot of SVN operations can be done from this interface (Figure 4).

Figure 4
 So if you are new to SVN, hope this will helpful to you.

Tuesday, December 20, 2011

Version Control Systems (Source/Revision Control Systems)

                                Version Control Systems are playing an important role in software source controlling. Most of the software industry used Version Control Systems to keep their source code in a central space so that any authorized person can access to it and do the changes in source files concurrently. The main purpose of a Version Control System is to enable collaborative editing and sharing of data, and keep the older states of the software code base (keep revisions).


                    In a project with large number of remote developers, there is a big probability to accidentally overwrite each other's changes in a regular file system (Figure 1). That's how the Version Control Systems come in to the picture in order to solve this problem.

Figure 1
                          So, to avoid above problem Version Control Systems used a strategy called Lock-Modify-Unlock model (Figure 2). This repository allows only one person to change specific file at a time. To do this it is using a lock system. The user who is going to change a file must lock the file so that any other user cannot lock the file again (it means file can be read but cannot change). After he made the change and commit the file to the repository he must unlock the particular file.

Figure 2
                                   But in this model, the developing process will be unnecessarily serialized, and by mistakes some users may not unlock a file, though others are waiting for unlocking that files. As well concurrent developing of two files by different users cannot have idea of each other file which have some dependencies between them.


                  Therefore another method called Copy-Modify-Merge model (Figure 3) is used as alternative to this problem. In here, users can concurrently develop the same file and commit it by after merging the new changes in repository to the working copy (locally changed copy).

Figure 3                                                                    
                            Version Control Systems such as SVN and CVS are using the above method to file managing.Using this method user have the full control of managing codes in conflicts (overwrite some code).

                  It's possible to classify the Version Control Systems as Centralized and Distributed. Centralized Version Control Systems means there is only one repository which developers are directly deals with. But in Distributed Version Control Systems are some kind of modification of the concept of Centralized Version Control Systems. The concept is to having two levels of Source Control Systems rather than one. So that, without all ways connecting with the main  Source Control System(main repository in server), using an own Source Control System(local host) until completing any specific work (can be a story of project). So that the main repository contains only working copies of the software. As well the operations (commit, revert) are faster due to using the own Source Control System more than Centralized Control System.


Some examples for Version Control Systems are as follows.

1). CVS (Concurrent Versioning Systems)
          Older technology. Simple system for make sure files and revisions kept up to date.

2). SVN (Subversion)
          Popular Version Control System. Most of the open-source projects use SVN.

3). git
          A fast populating, open-source Distributed Version Control System.

4). Mercurial
          Open-source Distributed Version Control System. Very faster and much simpler system than 'git'.

5). Bazaar
          Distributed Version Control System with different types of workflow than other Distributed Version Control Systems.

6). LibreSource
          Web portal, with a built-in subversion repositories. Useful for manage collaborative projects.

7). Monotone
          Fairly easy to learn. Places higher value on integrity than performance.


                               I think you may have some idea on Version Control Systems. And I'm willing to have another article soon on use of Subversion client.