For Web Developer

 open -a "Google Chrome" --args --disable-web-security

Solr commands - I tried

Start and stop solr
bin/solr start 
bin/solr stop 

Create and delete a core/collection
bin/solr create -c corename 
bin/solr delete -c corename

Index  a big TSV file
solr-7.5.0/bin/post -c gbif -params "separator=%09" -type text/csv ./gbif.tsv

Delete all documents from a core (Only documents) using curl command
curl "http://127.0.0.1:8983/solr/gbif/update?commit=true" -H "Content-Type: text/xml" --data-binary '<delete><query>*:*</query></delete>'

solr create and query sample

bin/solr start
bin/solr stop
bin/solr create -c corename
bin/solr delete -c corename

bin/solr create -c contacts
bin/post -c contacts mycontacts.xml

mycontacts.xml

<add> 
   <doc> 
      <field name = "id">001</field> 
      <field name = "first name">Rajiv</field> 
      <field name = "last name">Reddy</field> 
      <field name = "phone">9848022337</field> 
      <field name = "city">Hyderabad</field> 
   </doc>  
   <doc> 
      <field name = "id">002</field> 
      <field name = "first name">Siddarth</field> 
      <field name = "last name">Battacharya</field> 
      <field name = "phone">9848022338</field> 
      <field name = "city">Kolkata</field> 
   </doc>  
   <doc> 
      <field name = "id">003</field> 
      <field name = "first name">Rajesh</field> 
      <field name = "last name">Khanna</field> 
      <field name = "phone">9848022339</field> 
      <field name = "city">Delhi</field> 
   </doc>  
   <doc> 
      <field name = "id">004</field> 
      <field name = "first name">Preethi</field> 
      <field name = "last name">Agarwal</field> 
      <field name = "phone">9848022330</field> 
      <field name = "city">Pune</field> 
   </doc>  
   <doc> 
      <field name = "id">005</field> 
      <field name = "first name">Trupthi</field> 
      <field name = "last name">Mohanthy</field> 
      <field name = "phone">9848022336</field> 
      <field name = "city">Bhuwaeshwar</field> 
   </doc> 
   <doc> 
      <field name = "id">006</field> 
      <field name = "first name">Archana</field> 
      <field name = "last name">Mishra</field> 
      <field name = "phone">9848022335</field> 
      <field name = "city">Chennai</field> 
   </doc> 
</add>

queries
http://localhost:8983/solr/contacts/select?q=*:*
http://localhost:8983/solr/contacts/select?q=phone:9848022337

case insensitive by default
http://localhost:8983/solr/contacts/select?q=first_name:TrUpThi - 1 docs
http://localhost:8983/solr/contacts/select?q=first_name:TrUpTh - 0 docs
http://localhost:8983/solr/contacts/select?q=first_name:TrUpTh* - 1 docs

schema.xml / managed-schema / select a core > schema
<field name="test1" type="text" indexed="false" stored="false" required="false" />
Typically you will want your field to be either indexed or stored or both. If you set both to false, that field will not be available in your Solr docs (either for searching or for displaying). https://stackoverflow.com/a/22298265/6729230

Before adding docs to solr specify schema represented in schema.xml file. IF indexed or stored need is set false. Solr creates fields in schema based on the data and sets both true.
not advisable to change the schema after the documents have been added to the index. (http://www.solrtutorial.com/basic-solr-concepts.html)

Indexed and stored
While posting first solr analyse the data - analysis - lower casing, removing word stems, etc.
Analysis :  produces tokens. they are not original text, tokens are what are searched when you perform a search query. Indexed fields are fields which undergo an analysis phase and added to the indx. If a field is not indexed it cannot be searched. We cannot show tokens to users. we need to show the original text - there comes stored = true.
Sometimes, there are fields which aren't searched, but need to be displayed in the search results. You accomplish that by setting the field attributes to stored=true and indexed=false. Storing inceases the size of the index - index is analogous to database in MySQL.
Doubts

• what is the difference between managed-schema and schema.xml ?
• what is the difference between collection and core ?

Delete all documents in a collection - the collection name is gbif

JavaScript Callback

javaScript is the new software edge. It has a vivd scope in the mobile first, IoT first and AI first world. Programming paradigm has been shifted from sequential model to event based model. And javaScript tops in it. The fundamental most important part of javaScript is function express and callback.

Function expression
Unlike C or Java a function can be assigned to a variable in javaScript

function sum ( a, b ){
  return a+b;
}

is same as

var sum = function ( a, b ) {
  return a+b;
};

Function sum can be defined in any of the above ways. The second one is called function expression. The advantage of function expression is that since the function is assigned to a variable it can be passed as a parameter to another function.!!!

See the following example.

var sum = function ( a, b ) {
  return a + b;
};
var mul = function (a, b ){
  return a * b;
};

execute ( a, b, method ){
  return method (a, b);
}

console.log ( execute ( 3, 4, sum ) ); // prints 7
console.log ( execute ( 3, 4, mul ) ); // prints 12

In the above sample a method sum or mul is passed to another method execute. It may be a little difficult to understand.

Lets see another example:

greet ( name, cb ) {
  /** method implementations : if any **/
  cb ( 'Hello '+ name );
}

greet (  'Arun', function (msg){
  console.log ( msg );
});

The msg will be 'Hello Arun'. The idea is a callback is passed along with the function call. Inside the function after execution of method implementations, the call back is executed. We can use the return message inside our callback function.
Once this concept is clear,  it is very easy to expertise javaScript. 

A better way to pin point the delivery address.

One problem that is faced by ekart (Flipkart, Amazon, etc.) customers and the delivery people is to find the exact address of the customer. Especially in villages it is a big problem. Delivery people finds it difficult to track the home location. A solution is to provide a QR code of the exact delivery address (latitude, longitude) in the courier.Some updates need to be made in the purchase process. Along with the option for delivery address a map should also be provided on which the buyer can pin point the address. Google/Openstreet maps APIs are available to develop. For QR code also APIs are available. It is easy to integrate this idea to the purchase steps.

I tried to generate QR code of a random location manually. For that,

• Go to maps.google.com. Pin point a house >  right click and select 'what is here?' option. A box pops up on the bottom of the page as shown.

• Click  the number at the bottom which is the latitude and longitude of the address.
• Now click Share on left side menu and copy the link.

• I selected this location: https://goo.gl/maps/n52YtaxnSyG2
• To create a QR code for this location go to https://www.qr-code-generator.com/ and select URL option.
• Paste the location link and click on generate the QR code.
• That is it then it, now it just need to be printed on the courier. The full address can also be put on the QR code.

This is the QR code that I generated manually.

QR code can be decoded with a QR code scanner. Many apps are there in Playstore/Appstore. Some mobile phones has QR code scanner integrated to the camera app.

Cryptocurrency — the future of economy

Cryptocurrency a word we often hear in today's economic world. What is it? How is it evolved and what advantages it has over normal currencies.

Before diving into the details of cryptocurrency lets take a look at the economy that we live in. The currency that we use in our daily life is managed, printed and distributed by the governments through various banks. What is the problem with that ? Bank transactions are not open. Only the entities (person or organisations) involved and the bank has access to transaction details. Possibility of corruption is high there. Plenty of examples we have. Take the PNB case for instance. Illegal money transfer in huge quantities can’t be facilitated without the help from inside the banks. Massive financial leaks shakes the economy of a country.

Cryptocurrency is a solution. For a transparent economy what we need is a transparent ledger. Cryptocurrency is founded on this principle. The ledger is transparent, distributed and tamper proof.




Litecoin, Ethereum, Bitcoin, Monerio, Ripple, etc. are some cryptocurrencies available in the market. Take bitcoin for instance. One of the main technology (among may other) used to realize bitcoin is Blockchain.

Blockchain: Each bitcoin transaction is entered into a publicly distributed ledger. And all the transactions are verified and authenticated by cryptographic algorithms. The Ledger is distributed and is synchronized on multiple servers spread geographically apart.

The ledger which has enough entries (transactions) is called a block. It is then passed through an SHA algorithm.



SHA ( hash of previous block + current block + n ) = 0000000…0001010101101010…..0010101100101011

where ‘n’ is a big number which makes the first specific number of the result as zeros. The only way to find ‘n’ is sequentially checking one by one. This is the job of miners. They uses their IT infrastructure to find out this number and attach the total hash to the block. For their work they will get paid by some fraction of bitcoins from the issuers involved in the transactions on that block.


This block is then attached to the existing distributed and solved chain of blocks, hence the name blockchain. Once attached the transactions in that block in finished. The figure below gives a better idea on how blockchain is realised.

Since it is a distributed ledger anyone ( with proper access ) can trace back to all of the transactions that took place since inception. The cryptographic signatures makes the data tamper proof. Each hash calculation involved previous hash and therefore tampering this chain of information is impractical.

Cryptocurrency is the future. We are on the initial stages of an entirely new kind of economy. Latest trends in this space like IOTA confirms a bright future for it. Though with restrictions many countries legalised cryptocurrencies.

Mislead the boss

two files visit.sh and  links

#!/bin/bash

browser='Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0'

while IFS='' read -r line || [[ -n "$line" ]]; do
    curl -s -L -A ${#browser} $line
    sleep 5
done < "$1"

now populate the links

But it is sequentially finishing the links file - so make a random number generator and select a line from the links and make it while (1)

minify terminal default path

export PS1="(\W)$ "

A strategy for Google for a better Plus.

It has been a while since Facebook grabbed the social networking sovereignty from Google owned Orkut. Thenceforth google is persevering to recapture the realm. They withdrew Orkut and introduced a new platform there - the Google Plus. Unfortunately plus too did not work well.

So what is the problem?
Somebody famously quoted once - "The only way to be successful is to solve the pain or deliver a delight. Even solving a pain is the flip side of delivering a delight." Facebook already delivered that delight. Now what remains there for Google is to solve a problem ( if any ). Google Plus is a replica of Facebook in many ways - particularly how it works. Users has to have a better reason to switch to Plus when all their friends and family is in Facebook.

Again, start from the beginning.
As of now google is the most popular search engine. It has evolved at a time when there was a demand for an optimized as well as impartial search engine. Google literally vanquished other search engines of that epoch. All of that happened because google was solving a pain. Facebook is such a brilliantly designed application that it is difficult to find out a problem with it. It is a platform evolved gradually.

So how to conquer the platform back. The solution is to start from Google hangout - but one step at a time. Recently google made its hangout a separate unique web app google.com/hangouts which is a best place to kick off. Google has a user base which is much bigger than Facebook. 

• Hangout is not a popular word in some countries like India. It would be better to change it to a simple name. (How about the classic old name GTalk ?)
• Remove the hangout from Gmail and Plus and make it available only on GTalk app.
• Release clients for web, Chrome ext, Mozilla, android, iPhone and blackberry. 
• Let the users create a contact card in GTalk and let them set permissions to each field ( such as Public, Only me, Friends, Selected groups, Specific persons, Don't share with etc.), so that they can share their contact card to others who has GTalk installed.
• Let users create their public profile like http://gtalk.com/username for self branding. ( Like about.me ).
• Do not list strangers in GTalk search and no options in settings to enable it. This is very important.
• Searching strangers can be added in the timeline search.
• Never sync their personal contact with this app. Fetch user's contacts who has Gmail and suggest the user to add as friend in GTalk.
• List out GTalk contact details in a new address book.(Not google contacts) 
• Let the users create groups (open and closed).

Give time and let it flourish to a sophisticated chat application. 

Now comes the most important part of the social networking - the timeline. On almost all chat applications there is a provision to forward updates to another friend or group. Extend it to send it to your own time line. This is the idea. Share from chat to timeline. 

The timeline will have updates from our friends and family.

Now the application has three tabs namely chats, timeline and profile. While releasing make it available on all the platforms.

• Let users create pages and add authors - This will be useful for media. Only authors can post but any body can see it.
• Collaborate with mobile internet service providers to have a chat plan for a small amount ( something like Rs: 50 per month for 200MB ).

Current time in terminal - debian

Create a file 'now' and copy paste the following into it.

#!/bin/bash
date '+%A %D %X'

then save it and give executable permission to that file.

chmod +x ./now

Now copy that file to any of the system path. (echo $PATH gives the system paths)

cp ./now /usr/local/bin

that is it, next open a new terminal and type now and press enter. It will give the current time

Thursday 12/11/14 16:28:08

With a bit formatting we can have this printed in different colors.

#!/bin/bash

now=$(date '+%A %D %X')

red='\033[0;31m'

NC='\033[0m' # No Color

echo -e "${red}$now${NC}"

Check this for a list of color codes http://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux

Solve rubik's cube

Here is my demo,.

Solve the rubik's cube..
 * F' U L' U' - to make a white cross

White in bottom and other colors are opposite
  * R' 2D R D - brings white right then
  * R' D' R - do this if white is already right
  * D' R' D R

Middle layer: (top middles without yellow)
  * Top mid to right:
     U R U' R' U' F' U F
  * Top mid to left
     U' L' U L U F U' F'

Top Layer:
Ignore the corners and look at the center
* To create a cross:
  *  F R U R' U' F' which creates a  bar
  * try above until it creates a bar
  * then keep the bar horizondal
  * try the same above algorithm until you getcross

* Line up atleast two
  * Either line on side and back, or lines front and back
  * R U R' U R 2U R' U
  * Find out one that is lined up. It doesn't need to be oriented
  * Keep that in top right
  * U R U' L' U R' U' L - try until proper positions.
  * R' D' R D

Programming arduino uno with avr pocket programmer

Make the setup as shown in figure.

Image credits to https://learn.sparkfun.com/tutorials/installing-an-arduino-bootloader

1. Select 'Power Target' in 'AVR Pocket Programmer'.
2. git clone https://github.com/arunkumarv31/rtos
3. cd compile 
4. Edit Makefile. Comment avrdude line with option arduino and uncomment line with option usbtiny in burn section.
5. make
6. sudo make burn
7. sudo (or be root) is mandatory. Because linux may not show error as permission denied, instead it will show initialization failed, rc=-1. 

If you have your own code and hex file. Burn the hex with this command -

• sudo avrdude -F -V -c usbtiny -p atmega328p -D -b 115200 -U flash:w:main.hex 

This same procedure can be used with other boards.

For Arduino mega2560 sudo avrdude -F -V -c usbtiny -p atmega2560 -D -b 115200 -U flash:w:main.hex 

Edit inc/common.h and Makefile in rtos if you are using my codes.

upload bootloader to mega2560 using Uno

Run arduino ide (arduino 1.5.7 is required. won't work with older versions of software )
Connect the uno board to USB.
Open arduino ISP from examples.
Compile it and upload to uno board. (Board:Uno, Programmer Arduino as ISP)

Now make the following connections with the mega board. It is basically interfacing boards through SPI (MOSI, MISO, SCK) and SS(UNO) to RESET (MEGA2560)

UNO       MEGA2560
10 --------> 53
11 --------> 51
12 --------> 50
13 --------> 52

+5V --------> +5V
GND -------> GND

Select Board: Mega2560
Processor: 2560
Programmer: Arduino as ISP and then Tools > burn bootloader.Thsi time don't click upload.

Now connect mega2560 to PC and open a blink program. Compile and upload the program.

Install debian iso with usb

Creating a startup disk is simple in linux. No third party apps like start-up disk creator or unetbootin is required. Linux base command dd is enough.

• Download a debian version from this site https://www.debian.org/CD/http-ftp/#stable
• Mine is i386 so I downloaded http://cdimage.debian.org/debian-cd/7.5.0/i386/iso-cd/debian-7.5.0-i386-CD-1.iso
• Open a terminal and run sudo dmesg -c to clear kernel logs
• Connect the usb and then recheck dmesg in terminal (This time without -c option)
• there will be something like sdc sdc1 or sdbs db1 in the log.
• Replace the x with c or b in the following command and execute it. 
• dd bs=4M if=/path/to/archlinux.iso of=/dev/sdx && sync

That is it. A base kernel will get installed. While installing provide an internet connection so that it will automatically fetch required packages online. The installation will check for available connections ( Ethernet/WiFi ) while installing.

I think this works with almost all main linux distros

Ref: https://wiki.archlinux.org/index.php/Putting_installation_media_on_a_USB_key

Some tech reminiscence

Was formatting my PC and I got some old docs.
Btech final year project report: Secure Wireless Communication System
and
DESD ( CDAC ) seminar report: PageRank

An RTOS from the ground up

Just developed an RTOS from the ground up. It is still in the primitive stage but has got all the requirements of an RTOS. I used gcc toolchain to compile the sources and an Atmega328P (AVR) microcontroller to test it.

Source code is made available in my github. https://github.com/arunkumarv31/rtos

I have four infinite functions and the RTOS scheduler select the tasks to run and switches among them.
We can create tasks like this

createTask ( &function_1, "fun_one", 1, 200 );
 
createTask ( &function_2, "fun_two", 2, 200 );
.
.
createTask ( &function_n, "fun_three", n, 200 );

where the first parameter is the address of the function / task, the second parameter is its name. This name is used as the task identifier to change the attributes of a task, the third parameter is the priority of the task and the last parameter is the stack space required for the task.

A user function / task can be something like this. usually it will be an infinite loop.

void function_2 ( void )
{
  while (1)
  {
      changePriority ( "fun_one", 3 );
      changePriority ( "fun_two", 5 );
      changePriority ( "fun_three", 2 );
      changePriority ( "fun_four", 4 );
     
      printf ( "function_2\n" );
  }
}

The scheduler follows "fixed priority pre-emptive scheduling". And it always executes the highest priority task. So to give multitasking effect, i.e to preempt the current task and to execute other tasks we need to change the priority of the other tasks  in the current task with changePriority( "fun_one", 2 ) method.

If two or more tasks has the same priority which is the highest priority then ithe scheduler will execute the task first created.

changeStatus ( "fun_three", WAIT ) is used to change the status of a task. Here I have three status RUN, TERMINATE and WAIT.

deleteTask ( "fun_two" ) will delete the task and it will no longer be available to execute.

Saving SP in atmega328p

Context switching is one of the core component of parallel processing. I was trying to implement one such. Storing the SP to data space is something noteworthy.

Create a printf logic in arduino uno for debugging - avr-gcc

If you are using arduino uno board with avr-gcc compiler. You can have a printf printing system for better debugging.

Clear your memory card safely

Deleting a file in Win/Linux is almost like locking in a safe. It can be  retrieved from the drive easily with a recovery program as long as it is not overwritten by some other data.

Install simulavr from the source

An apt-get install simulavr will install a previous version of simulavr in Debian Wheezy which has no support for 'atmega328'. So it is better to install it from the latest source.