How to implement system command using python


In this post I will show you that how you can exactly implemented system command using python.Here we use for example ls -l command. When I use this command in terminal it gives me following output

Harshs-MacBook-Pro:scripts harshkothari$ ls -l
total 32
-rw-r--r--  1 harshkothari  staff    13 Jul 23 02:33 demo.ini
-rw-r--r--@ 1 harshkothari  staff  1864 Jul 23 04:30 filesread.py
-rw-r--r--  1 harshkothari  staff  4121 Jul 23 04:30 langtofontmap.json

Now this command we can implement by following methods using python 🙂

  • os.system("command with args") passes the command and arguments to your system’s shell. This is nice because you can actually run multiple commands at once in this manner and set up pipes and input/output redirection.
    os.system("command < input_file | anothercommand > output_file")
    However, while this is convenient, you have to manually handle the escaping of shell characters such as spaces, etc. On the other hand, this also lets you run commands which are simply shell commands and not actually external programs. http://docs.python.org/lib/os-process.html

    >>> import os
    >>> os.system('ls -l')
    total 32
    -rw-r--r--  1 harshkothari  staff    13 Jul 23 02:33 demo.ini
    -rw-r--r--@ 1 harshkothari  staff  1864 Jul 23 04:30 filesread.py
    -rw-r--r--  1 harshkothari  staff  4121 Jul 23 04:30 langtofontmap.json
    0
  • stream = os.popen("command with args") will do the same thing as os.systemexcept that it gives you a file-like object that you can use to access standard input/output for that process. There are 3 other variants of popen that all handle the i/o slightly differently. If you pass everything as a string, then your command is passed to the shell; if you pass them as a list then you don’t need to worry about escaping anything. http://docs.python.org/lib/os-newstreams.html
    >>> import os
    >>> print os.popen('ls -l').read()
    total 32
    -rw-r--r--  1 harshkothari  staff    13 Jul 23 02:33 demo.ini
    -rw-r--r--@ 1 harshkothari  staff  1864 Jul 23 04:30 filesread.py
    -rw-r--r--  1 harshkothari  staff  4121 Jul 23 04:30 langtofontmap.json
  • The Popen class of the subprocess module. This is intended as a replacement for os.popenbut has the downside of being slightly more complicated by virtue of being so comprehensive. For example, you’d say
    print Popen("ls -l", stdout=PIPE, shell=True).stdout.read()
    instead of
    print os.popen("ls -l").read()
    but it is nice to have all of the options there in one unified class instead of 4 different popen functions. http://docs.python.org/lib/node528.html

    >>> from subprocess import Popen, PIPE
    >>> print Popen("ls -l", stdout=PIPE, shell=True).stdout.read()
    total 32
    -rw-r--r--  1 harshkothari  staff    13 Jul 23 02:33 demo.ini
    -rw-r--r--@ 1 harshkothari  staff  1864 Jul 23 04:30 filesread.py
    -rw-r--r--  1 harshkothari  staff  4121 Jul 23 04:30 langtofontmap.json
    
  • The call function from the subprocess module. This is basically just like the Popen class and takes all of the same arguments, but it simply wait until the command completes and gives you the return code. For example:
    call(["ls", "-l"], shell=True) http://docs.python.org/lib/node529.html

    >>> from subprocess import call
    >>> call(["ls", "-l"], shell=True)
    demo.ini                filesread.py            langtofontmap.json
    0
  • Using command module http://docs.python.org/2/library/commands.html
    >>> import commands
    >>> commands.getstatusoutput('ls -l')
    (0, 'total 32\n-rw-r--r--  1 harshkothari  staff    13 Jul 23 02:33 demo.ini\n-rw-r--r--@ 1 harshkothari  staff  1864 Jul 23 04:30 filesread.py\n-rw-r--r--  1 harshkothari  staff  4121 Jul 23 04:30 langtofontmap.json')
    >>> commands.getoutput('ls -l')
    'total 32\n-rw-r--r--  1 harshkothari  staff    13 Jul 23 02:33 demo.ini\n-rw-r--r--@ 1 harshkothari  staff  1864 Jul 23 04:30 filesread.py\n-rw-r--r--  1 harshkothari  staff  4121 Jul 23 04:30 langtofontmap.json'
    
  • The os module also has all of the fork/exec/spawn functions that you’d have in a C program

Hope this will help you 🙂

Pass Random variable to shell script rather then $1 or $2 using php


Here I am showing you how can you pass the random variable to shell script. ( i.e like $uname 🙂 )

You can receive data using $1 or $2 variable but you can’t directly receive data into $uname or any random variable.
Here is my bash file demo.sh

#!/bin/bash
echo $uname

Now I have to pass $uname variable so directly using this php function you cant do this

exec("sh demo.sh harshkothari",$output);

You have to send it via environment variable . This is simple code snippet how you can you pass this. php file name is shell.php

$uname = 'harshkothari';
putenv('uname='. $uname); //put environment variable uname 
exec("sh demo.sh",$output); //execute command
echo($output[0]. "\n"); //print return value

$output will return the last line of output and it will show like this

Harshs-MacBook-Pro:lcmapiuse harshkothari$ php shell.php 
harshkothari
Harshs-MacBook-Pro:lcmapiuse harshkothari$ 

Here is snapshot of terminal output

Output of php script, passing random variable to shell script

Output of php script, passing random variable to shell script

Wikimedia Tool setup for LCM-Dashboard (GSoC Project)


Wikimedia tools login screen using ssh

Wikimedia tools login screen using ssh

wikimedia tool setup is very tedious work but #wikimedia-labs helps a lot.

After creating an account and being approved by admin. You will log in to your tool using this command.

ssh -A username@tools-login.wmflabs.org

After logging into your tool you have to use following command to switch to your service

become toolname

Now this have following things

  • public_html – where you can place your html, php, css, js

  • cgi-bin – where you can place your python script etc

  • replica.my.cnf – contains username and password

Now Creation of databse for your project

In the first step you have to find you username and password by using “” file.

After that your database name must be USERNAME__DBNAME (2 underscore)

mysql --defaults-file="${HOME}"/replica.my.cnf -h tools-db 

after that MariaDB console will open and in that you can create databasw

MariaDB [(none)]> CREATE DATABASE USERNAME__DBNAME;

you can access this database by using this command

mysql --defaults-file="${HOME}"/replica.my.cnf -h tools-db USERNAME__DBBAME

If you want to connect the database to your project

host = tools-db
uname = your_username
pass = your_password
dbname = USERNAME__DBNAME

Then you can push the file to remote using git via clone or you can use scp command to push files from remote to local like this

scp <files> USERNAME@tools-login.wmflabs.org:/data/project/<tool-name>

Hope it will help you 🙂

Link of my tools on wikimedia labs : lcm-dashboard

Being Google Summer of Code Intern


I am very excited (and also a little late :D) to share the news that I have been selected as an intern for Google summer of code in Wikimedia Foundation for the project of developing language coverage matrics dashboard for language engineering team of Wikimedia Foundation.

I would like to thank my mentors Runa Bhattacharjee and Alolita Sharma, for guiding me throughout and very special thanks to Amir Aharohi for his valuable inputs for this proposal. Last but not the least special thanks to Sumana and Quim for polishing up my proposal, and the feedbacks all this while.

Being selected as a GSoC intern was finally a good news after quite while. The result came at 12:30 midnight – I couldn’t sleep. Nevertheless, great night it was and so was the news.

Community bonding period started right after. I tinkered around a few new technologies and did the overall planning for my project.

Until June 17

Until June 24

  • Created new language entry system

    Search + New language button

    Search + New language button

    Language Entry form

    Language Entry form

     

  • Created language search system with suggestion

Language search with suggestion facility

Language search with suggestion facility

Language Search result

Language Search result

 

Until July 1

  • created and set up primary thing on wikimedia labs

  • jquery.ime implemented in project

  • done minor changes in language search system

  • created on the spot editing facility for any language detail under admin privileges

Edit button for on the spot editing

Edit button for on the spot editing

when click on edit button this menu will appear

when click on edit button this menu will appear

After saving the changes it will change

After saving the changes it will change

Until July 8

  • set up basic filtering functionality

  • set up some test cases to find smooth and fast filter facility

  • created simple idea of API for language data

Until July 15

  • Created Filtering functionality : By this time, a filtering functionality seemed to be an important feature to have on the LCMD interface. Filtering simply means that there would be a list of the language tools on one side along with checkboxes and the checking those would output the list of the languages that have those tools enabled. So, if say suppose, we have both jquery.ime and jquery.webfonts checked in, the language output will give a list of the common languages with both the tools enabled. This was kind of important. Screen shot and more detail on another blog post.

  • Set up tools instance for project tool instance

  • Set up new repo for project with lots of commits New repo

  • Created simple php API for language data :  Here I have developed a simple prototype version of Language detail APIs so that any website / user want to use that information that can fetch the details using that API

Return dataType is : JSON and if you are using cross domain data transfer then it should be JSONP.


$.ajax({
    url : 'http://tools.wmflabs.org/lcm-dashboard/lcmd/api/php/dataapi.php?query=jquery_ime',
    dataType : 'JSONP',
    type : 'GET',
    //data : 'query=webfonts jquery_ime' ,
    success : function(data){
        console.log(data);
    },
    error : function(data){
        console.log(data);
    })

It will return the data in JSON format.

Beautiful example of this API usage is here

http://mikipedian.blogspot.in/2013/07/i-see-rather-what-all-i-now-visualize.html

 

OpenCV on your MAC – Easy to install and Configure


OpenCV

OpenCV

OpenCV (Open Source Computer Vision Library) is a library of programming functions mainly aimed at real-time computer vision, developed by Intel, and now supported by Willow Garage and Itseez. It is free for use under the open source BSD license. The library is cross-platform. It focuses mainly on real-time image processing. If the library finds Intel’s Integrated Performance Primitives on the system, it will use these proprietary optimized routines to accelerate itself.

Here in this BlogPost I will explain you to install OpenCV(version 2.3, 2.4) on your MAC 🙂

There are two way to build openCV on your MAC.

1. You can use MacPorts

For this first you have to install MacPorts. The easiest way to install MacPorts on a Mac OS X system is by downloading the dmg for Mountain LionLionSnow Leopard orLeopard and running the system’s Installer by double-clicking on the pkg contained therein, following the on-screen instructions until completion. Installation package is here.

After installing MacPort you have to update it to latest version.

$ sudo port selfupdate

Now you can install OpenCV by running following command

$ sudo port install opencv

If you want to install python building

$ sudo port install opencv +python27

or with python 2.6

$ sudo port install opencv +python26

or with QT

$ sudo port install opencv +qt4

So by this way you can install OpenCV in your MAC OS.

2. You can use CMake

By using CMAKE you can also install OpenCV on  your mac. For that first you should have to install CMake, which thankfully comes with Mac binaries – so that was easy. You can download .dmg file here.

After installing CMake you have to download the OpenSource Package that you can download by clicking here.  Now you have OpenCV-2.4.3 ter.bz2 and unzip to folder. Now open terminal and go to that folder and write following commands in Terminal.

$ mkdir build

$ cd build

$ cmake -G "Unix Makefiles" ..

$ make -j8

$ sudo make install

Now You have done. Congrats you have installed OpenCV on your MAC 🙂

Now you can check by doing following thing.

GO to TerminalType Python

Now type

>>> import cv

Now if it run without any error then \m/ but if you encountered with following error

>>> import cv
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
ImportError: No module named cv

Then close python by pressing ctrl + D key and now you have to run following command

 $ export PYTHONPATH=/usr/local/lib/python2.7/site-packages/

after running this command restart the Terminal and then open python and import cv this will work 🙂

If you have any doubt or problem then you can comment here. Thanks.