Wednesday, 23 October 2013

Tips for IntelliJ IDEA


Color Themes

Some of my favourite themes are

1. clone intellij-colors-solarized
$ git clone https://github.com/jkaving/intellij-colors-solarized.git
2. Goto File | Import Settings... and specify the intellij-colors-solarized directory
3. Goto Preferences | Editor | Colors & Fonts and select one of the new color themes.




2) Spacegray



3) classic eclipse -
http://color-themes.com/?view=theme&id=563a1a6b80b4acf11273ae67


http://color-themes.com/?view=theme&id=5b491e3c50544f1700232dbc



4) For, Monokai color scheme

5) For Twilight,
wget --directory-prefix=~/.IntelliJIdea13/config/colors/ https://raw.githubusercontent.com/eed3si9n/color-themes/master/IntelliJ-IDEA/Twilight/Twilight.xml


setup auto imports


[2] Editor 
[2.1] Auto scroll from source


OR add ProjectPane=true to ~/.idea/workspace.xml



[2.2] Show line numbers



[2.3] Disable autosave
Goto Settings and search "Synchro", and go to General inside of IDE Settings.




Then, Enable marking modified files with *





Optimize imports on the fly







[3] Keymap
[3.1] Delete a line : C-k from the cursor point (emacs keymap) or C-y in default

C-a C-k to delete a line


[3.2] @Override methods - C-o




[3.3] add a breakpoint - C-f8

[3.4] C Shit f12  http://stackoverflow.com/a/10990239/432903



[3.4] change keymap to emacs



[3.5] column width (<=100)




[4] change file header




Windows/ Buffers 

Switch between multiple intellij windows/projects

C-x o

Switch the buffer

C-x p/n

Kill the buffer

C-x k

Vertical selection

Alt- then drag cursor up or down

[5] Run Main class with app options




References

https://docs.google.com/document/d/1kdUrsjCOIBunzk5OYBojL-58LNguFTLl_KZqNIva3tg/edit#heading=h.bnc6aa9l7gnu

What’s Cool In IntelliJIDEA. Part I, http://refcardz.dzone.com/refcardz/intellij-idea
 ,http://sethmason.com/2007/06/29/ten-keyboard-shortcuts-for-intellij-idea.html
 ,http://arhipov.blogspot.com/2011/06/whats-cool-in-intellijidea-part-i.html
http://stackoverflow.com/a/14495004/432903

http://stackoverflow.com/a/2648361/432903

Monday, 21 October 2013

First night hack on elasticsearch (3 hrs)

ElasticSearch is (Apache Solr like) Lucene based distributed RESTful (PUT, GET, POST, DELETE) search server developed in Java.
To get motivated read a soundcloud casestudy and watch Search and Discovery at SoundCloud.

Elasticsearch in 15 minutes from David Pilato is a nice resource to follow.





Following Slide is also a useful one(with Rails tutorial - slides#44).



Now, Let's get hands dirty.

[STEP 1] download and tar elasticsearch 0.90.5
prayag@prayag:~$ wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.5.tar.gz

prayag@prayag:~$ tar -zxvf elasticsearch-0.90.5.tar.gz

[STEP 2] start elasticsearch node (in foreground)
prayag@prayag:~$ elasticsearch-0.90.5/bin/elasticsearch -f
[2013-10-21 23:19:19,127][INFO ][node                     ] [Aardwolf] version[0.90.5], pid[15897], build[c8714e8/2013-09-17T12:50:20Z]
[2013-10-21 23:19:19,128][INFO ][node                     ] [Aardwolf] initializing ...
[2013-10-21 23:19:19,140][INFO ][plugins                  ] [Aardwolf] loaded [], sites []
[2013-10-21 23:19:23,523][INFO ][node                     ] [Aardwolf] initialized
[2013-10-21 23:19:23,524][INFO ][node                     ] [Aardwolf] starting ...
[2013-10-21 23:19:23,800][INFO ][transport                ] [Aardwolf] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.195.8:9300]}
[2013-10-21 23:19:27,045][INFO ][cluster.service          ] [Aardwolf] new_master [Aardwolf][pK-h7KRwTWamnQNNrR2gwQ][inet[/192.168.195.8:9300]], reason: zen-disco-join (elected_as_master)
[2013-10-21 23:19:27,143][INFO ][discovery                ] [Aardwolf] elasticsearch/pK-h7KRwTWamnQNNrR2gwQ
[2013-10-21 23:19:27,233][INFO ][http                     ] [Aardwolf] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.195.8:9200]}
[2013-10-21 23:19:27,234][INFO ][node                     ] [Aardwolf] started


[2013-10-21 23:19:27,311][INFO ][gateway                  ] [Aardwolf] recovered [0] indices into cluster_state

Verify elasticsearch is running at 9200, 
First You may need to install curl ,
sudo apt-get update
sudo apt-get install curl

Then, curl following request in terminal,
$ curl -XGET 'http://localhost:9200'
{
  "ok" : true,
  "status" : 200,
  "name" : "Lady Octopus",
  "version" : {
    "number" : "0.90.5",
    "build_hash" : "c8714e8e0620b62638f660f6144831792b9dedee",
    "build_timestamp" : "2013-09-17T12:50:20Z",
    "build_snapshot" : false,
    "lucene_version" : "4.4"
  },
  "tagline" : "You Know, for Search"
}

[STEP 3] create index movies

Then, put following script in create_index.sh, and then execute the same.
curl -XPUT 'http://localhost:9200/movies/'

$ bash create_index.sh

[STEP 4] apply mapping for type 'Movie'
Put following script in movie_mapping.sh, and then execute the same.

curl -X PUT localhost:9200/movies/Movie/_mapping -d '{
    "Movie" : { "properties"  : { 
                    "title"    : { "type":"String" }, 
                    "director" : { "type":"String" }, 
                    "year"     : { "type":"long" }
                }
              }
}'

$ bash movie_mapping.sh


[STEP 5] Indexing(create/update) documents 
$ curl -XPUT http://localhost:9200/<index>/<type>/<id>,
where type(in ES) = table (in RDBMS)

Put following script in movie_document.sh, and then execute the same.
curl -XPUT 'http://localhost:9200/movies/Movie/1' -d '
{
    "title": "The Godfather",
    "director": "Francis Ford Coppola",
    "year": 1972
}'

$ bash movie_document.sh

Response we get is, 
{"ok":true,"_index":"movies","_type":"Movie","_id":"1","_version":1}

And, check the elasticsearch console with following updates.
[2013-10-22 00:12:46,835][INFO ][cluster.metadata         ] [Aardwolf] [movies] creating index, cause [auto(index api)], shards [5]/[1], mappings []
[2013-10-22 00:12:47,944][DEBUG][action.index             ] [Aardwolf] Sending mapping updated to master: index [movies] type [movie]
[2013-10-22 00:12:47,961][INFO ][cluster.metadata         ] [Aardwolf] [movies] update_mapping [movie] (dynamic)


[STEP 6.1] Retrieve document by id
Syntax : 
$ curl -XGET http://localhost:9200/<index>/<type>/<id>

Execute following command to retrieve movie with document id 1,
$ curl -XGET "http://localhost:9200/movies/Movie/1?pretty=true"
{
  "_index" : "movies",
  "_type" : "Movie",
  "_id" : "1",
  "_version" : 1,
  "exists" : true, 
"_source" : 
{
    "title": "The Godfather",
    "director": "Francis Ford Coppola",
    "year": 1972 
}
}

[STEP 6.2] Search documents by type field (title)
$ curl -XGET "localhost:9200/movies/Movie/_search?q=title:Godfather"

 {"took":22,
"timed_out":false,
"_shards":{"total":5,"successful":5,"failed":0},
"hits":{
"total":1,
"max_score":0.30685282,
"hits":
[{"_index":"movies",
"_type":"Movie",
"_id":"1",
"_score":0.30685282, 
"_source" : 
{
    "title": "The Godfather",
    "director": "Francis Ford Coppola",
    "year": 1972 
}
}]
}
}


OR Using Elasticsearch Query DSL, 





[7] Not to miss 
[7.1] 40 mins presentation by +Shay Banon 


                                   Shay Banon - ElasticSearch: Big Data, Search, and Analytics

The presentation includes following topics : 
data design patterns
The "kagillion" shards problem
Simple data flows
"Users" data flow
"time" data flow (clicks etc)
more than search
questions?
etc

[7.2] ElasticSearch, "You know, for search", Presentation by Clinton Gormley
Also watch this guy at "Getting down and dirty with Elasticsearch"





[7.3] book exploring elasticsearch, Andrew Cholakian



References
ElasticSearch 101– a getting started tutorial, http://joelabrahamsson.com/elasticsearch-101/

ElasticSearch in 5 minutes, http://www.elasticsearchtutorial.com/elasticsearch-in-5-minutes.html

http://blog.florian-hopf.de/2013/09/simple-event-analytics-with.html

http://www.javacodegeeks.com/2013/04/getting-started-with-elasticsearch.html

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-health.html

http://thediscoblog.com/blog/2013/09/03/effortless-elasticsearch-clustering/

Scaling massive elastic search clusters - Rafał Kuć - Sematext, http://www.slideshare.net/kucrafal/scaling-massive-elastic-search-clusters-rafa-ku-sematext



Shards and replicas in Elasticsearch, http://stackoverflow.com/a/15705989/432903

Friday, 11 October 2013

Existence of Mathematics


TRICK 1
http://www.youtube.com/watch?v=pWtF94cTZ6k


The mathematics behind is
STEP 1 The piles are 
Pile 1 : 10
Pile 2 : 15
Pile 3 : 15
9

STEP 2 
(Pile 1)+1(+x from Pile2)
(Pile 2)=y+1+(x’ from Pile3)
(Pile 3)=y’+1+9

STEP 3
Now, the whole deck becomes
Deck = (10+1)+(x+y+1)+(x’+y’+1)+9

Since, x+y =15(Pile2) and x'+y' =15(Pile3)
Deck = (10+1)+(15+1)+(15+1)+9
Deck = 11(position1)+16(position2)+16(position3)+9

From the bottom, positions of selected cards is,
(11, 27, 43) => (38, 22, 6) from the top.

Adding 4 cards to the bottom, positions of selected cards become
(15, 31, 47)

STEP 4
Uping and Downing, causes the cards to be left at even position ie the Downs, with following positions :
(15, 31, 47)
(  8, 16, 24),
(  4,   8, 12),­
(  2,   4,   6),
(  1,   2,   3)

TRICK 2



References
http://www.youtube.com/results?search_query=mathematics+card+trick&oq=mathematics+card+&gs_l=youtube.3.0.0l2j0i10l3j0i5i10l5.7781.20275.0.22316.17.16.1.0.0.0.457.3182.1j8j6j0j1.16.0...0.0...1ac.1.11.youtube.Chhz7mOAxu4


Saturday, 5 October 2013

Understanding Unix - Part 2 (for programmers)

This post is a sequel of previous post Linux Hacks Part 1 mostly important for programmers.


hack 1 => copy text to the clip(board)
Copy text from file Service.groovy,
$ xclip -selection clipboard grails-app/domain/eccount/Service.groovy

Paste to a new file ServiceNew.groovy
$ vi grails-app/domain/eccount/ServiceNew.groovy

Go INSERT mode and Ctrl+Shift+V

OR

gg"*yG

Reference  :
xclip Does Copy-and-Paste on the Linux Command Line, http://www.linuxplanet.com/linuxplanet/tips/6788/1
VI editor: Copy all the lines to clipboard,  http://stackoverflow.com/a/1620029/432903

hack 2 => find files with a name inside a directory
find searches in the real system.
is slower but always up-to-date and has more options (size, modification time,...)

eg. 1
loohcs$ find . -name "*Service*"
./test/unit/eccount/StudentServiceTests.groovy
./test/unit/eccount/UserServiceTests.groovy
./grails-app/domain/eccount/Service.groovy
./grails-app/domain/eccount/ServiceTransaction.groovy
./grails-app/services/eccount/TransactionService.groovy
./grails-app/services/eccount/UserService.groovy

eg. 2
$ find /packup/repo.softwares/ -name "*cassandra*"
/packup/repo.softwares/JVM/SolrLuceneES-BigData/apache-cassandra-2.0.6-bin.tar.gz
/packup/repo.softwares/JVM/SolrLuceneES-BigData/apache-cassandra-2.0.7-bin.tar.gz

Reference 10:
10 find command in unix examples basic, http://javarevisited.blogspot.com/2011/03/10-find-command-in-unix-examples-basic.html

What is the different between 'locate' and 'find' in Linux?, http://superuser.com/a/199473

hack 3 => awk (intrepreted prog lang)
find the total number of lines in a file (without using NR)
prayag@prayag:~/hacker_/loohcs$ awk 'BEGIN {sum=0} {sum=sum+1} END {print sum}' grails-app/controllers/LoginController.groovy 
134

hack 4 => SED (Stream EDitor
4.1) replace the word "ServiceTransaction" with "Transaction" in file


Stream editor is Stream editor not file editor.

root@0cad383c1249:/# sed --version
sed (GNU sed) 4.4
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Jay Fenlason, Tom Lord, Ken Pizzini,
and Paolo Bonzini.
GNU sed home page: <http://www.gnu.org/software/sed/>.
General help using GNU software: <http://www.gnu.org/gethelp/>.
E-mail bug reports to: <bug-sed@gnu.org>.

prayag@prayag:~/hacker_/loohcs$ sed s/ServiceTransaction/Transaction/ < TransactionService.groovy

## OR 
$ echo "Mbr Hlth Screening Mammogram" | sed s/Hlth/Health/ | sed s/Mbr/Member/
Member Health Screening Mammogram


## OR replace a word with special character containing $
sed --in-place 's/shaharma/\$ORGANISATION_NAME/' deployment.sh

4.2) delete empty lines
$ sed '/^$/d' src/main/resources/MoneyReply.xml > src/main/resources/MoneyReplyOut.xml

Practice more at Sed Interview Questions

4.3) remove certain chars

## Following is a script to remove character ) 
$ sed 's/[),]//g' Diagnosis.html

## remove \ character in serialised json,
sed 's/[\\]//g' < data

4.4) replace a line with certain content
Syntax -
sed -i '/TEXT_OF_A_LINE_TO_BE_REPLACED/c\prayagupd just changed this line.' FILE_NAME

eg.
sed --in-place '/Dvbox/c\   "-Dvbox.home=/usr/lib/virtualbox"' project.clj

on macos,

sed -i.bu 's/Dvbox/yo/' project.clj

grep using sed
sed -n '/Chrome/p' < apple.log | tee newApple.log

hack 5 => ssh connect
$ ssh -v uatestinguser@10.13.222.123

Copy file from remote linux server to local machine
$ scp -v uatestinguser@10.13.222.250:/tmp/analytics.csv /home/prayag/

Reference reading : 
Section 1.2: How do SSH, Telnet and Rlogin differ?,  http://cc-ipcp.icp.ac.ru/Section1.2.html
The SSH (Secure Shell) vs RemoteLogin Protocol, http://www.slideshare.net/sourav894/ssh-and-rlogin
3.3 Setting up permissions for rsh, rlogin and rcp(.rhosts), http://rmni.iqfr.csic.es/guide/man/lynux/chap3-3.htm

hack 6 > ls 
list files sorted by date in desc order
$ ls -tr

hack 7 => length of a string

(length of Long.MAX_VALUE in java)

echo "9223372036854775807" | wc -c

      20

$ expr length '$2a$10$osjPAbnn9ogMsY7z.Q7rWe2FLTx/C9Yi6.mj74aTbXWC3iyR1p/0q'
60

hack 8 => find a file with particular text 
The grep syntax looks as below.
$ grep -r "<SEARCH_TEXT>" <DIRECTORY_PATH>

$ grep -r "navbar-merchant" nioc2egdelonk/
nioc2egdelonk/web-app/less/nav.less:.navbar-admin {

hack 9 : know gtk version
$ dpkg -l libgtk[0-9]* | grep ^i
ii  libgtk2-perl                           2:1.223-1build3                         Perl interface to the 2.x series of the Gimp Toolkit library
ii  libgtk2.0-0                            2.24.10-0ubuntu6                        GTK+ graphical user interface library
ii  libgtk2.0-bin                          2.24.10-0ubuntu6                        programs for the GTK+ graphical user interface library
ii  libgtk2.0-common                       2.24.10-0ubuntu6                        common files for the GTK+ graphical user interface library

hack 10 : install gtk-recordmydesktop
$ sudo apt-get install gtk-recordmydesktop

Configure laptop settings as mentioned at recordMyDesktop output video shows *massive* artifacts, else plays too fast lowering resolution to to 1360x768.
xrandr -s 1360x768



Start recordmydesktop 
$ gtk-recordmydesktop -o /home/prayag/Videos/gwt_screencast.ogv

Convert *.ogv to *.flv with same-quality (size will increase though)
$ ffmpeg -sameq -i /home/prayag/Videos/gwt_screencast.ogv /home/prayag/Videos/gwt_screencast.flv

hack 11 : Zip a file
$  zip -r eccount-hg *



References
How can I create launchers on my desktop?, http://askubuntu.com/a/64237/37643
Csv to Html Table, https://gist.github.com/prayagupd/8050493

Thursday, 3 October 2013

Understanding MySQL Joins




References
What's the difference between INNER JOIN, LEFT JOIN, RIGHT JOIN and FULL JOIN, http://stackoverflow.com/a/6188334/432903