Add A New Local IP Address in FreeBSD

Before you upload an internet site, you better test it on your local machine. To do that, you should allocate an IP address known as a loopback address that does not require a modem for access. If you’ve installed Apache Httpd server, you’ll probably get an HTML page that reads “It works” upon connecting to “http://localhost” or http://127.0.0.1” from the web browser. But what if you want to create another site? How to make your server recognize an IP?
In this post I will describe by example the process ot adding a local IP.

Step 1: Associate an IP with a Domain Name

If you want to create a domain name such as ‘example.coq‘, add a line for it in /rc/hosts in the format:

<inet-addr>   <alias>

For example:

127.0.0.2               example.coq

Step 2: Attach the IP Address To a Network Interface

To make an address available to internet servers, attach it to a network interface.
A network interface is the identifier followed by colons at the beginning of blocks returned by the command ifconfigFor example:
In this block lo0 is an interface name.

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128 
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
    inet 127.0.0.1 netmask 0xff000000 

From the prefix ‘lo’ of the interface name, you can know it is a loopback interface.

You can attach an address, login as root, and run ifconfig like in the following example:
ifconfig lo0 127.0.0.2 netmask 255.255.255.255 alias
This will attach IP address 127.0.0.2 to the loopback interface ‘lo0’

You can make the operating system add it each type you start your computer, by adding the following line to /etc/rc.conf:

ifconfig_lo0_alias0="inet 127.0.0.2 netmask 255.255.255.255"

You can learn more about virtual hosts from the section ‘11.6 Virtual Hosts” of the FreeBSD Handbook

Step 3: Start a Listening Server

Now, you can start a server that will listen on your address. You can do it by adding a virtual host in apache httpd, create a server in ‘node.js’, etc.
If you’ve installed ‘Apache24’ from the ports, you can find documentation in '/usr/local/share/doc/apache24'. In addition, you can find documentation in the httpd site.

Written with StackEdit.

Advertisements

Implementing Web Sockets With cURL

WebSocket is an internet protocol that allow full-duplex communication between a client and a TCP/HTTP server. This means that data can be passed in both directions simultaneously. Unlike HTTP, in WebSocket protocol, the client doesn’t have to send a request in order to get responses. Incoming messages are handled by event handlers. HTML5 supports the javascript object WebSocket.  This object has the function ‘send’ that sends a message to the server, and event listener defined by the developer:

onOpen(evt) – Connection Opened.

onMessage(evt) – Message received.

onError(evt) – Error occured.

onClose – Connection Closed.

A little example can be found here.

How to do it with cURL?

cURL does not support the WebSocket protocol. It won’t process a request if the URL string begins with ‘ws://’. You should use ‘http://&#8217; or ‘https://&#8217; instead, and define request headers and cURL event listeners yourself. In cURL, you’ll define:

  •  A header function using the option ‘CURLOPT_HTTPHEADER’ to check if a connection has been established.
  •  A write function using ‘CURLOPT_WRITEFUNCTION’ to handle in-coming messages.
  •  A socket function using CURLOPT_OPENSOCKETFUNCTION to obtain an IO socket thru which to send messages to the server.

The WebSocket protocol is standardized by RFC 6455,

The following sections will discuss the parts of a C program implementing a Web Socket.

The Main Loop

This part creates a cURL handle by calling curl_easy_init(), defines headers, URL and callback functions using curl_easy_setopt(), and finally call ‘curl_easy_perform().

Following is an example:

  handle = curl_easy_init();
  // Add headers
  header_list_ptr = curl_slist_append(NULL , "HTTP/1.1 101 WebSocket Protocol Handshake");
  header_list_ptr = curl_slist_append(header_list_ptr , "Upgrade: WebSocket");
  header_list_ptr = curl_slist_append(header_list_ptr , "Connection: Upgrade");
  header_list_ptr = curl_slist_append(header_list_ptr , "Sec-WebSocket-Version: 13");
  header_list_ptr = curl_slist_append(header_list_ptr , "Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==");
  curl_easy_setopt(handle, CURLOPT_URL, "http://echo.websocket.org");
  curl_easy_setopt(handle, CURLOPT_HTTPHEADER, header_list_ptr);
  curl_easy_setopt(handle, CURLOPT_OPENSOCKETFUNCTION, my_opensocketfunc);
  curl_easy_setopt(handle, CURLOPT_HEADERFUNCTION, my_func);
  curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, my_writefunc);
  curl_easy_perform(handle);

Obtaining a Socket

The socket is a resource used for sending messages to the server. Messages received from the server will be handled by the write function. Following is a function that returns the desired socket:

curl_socket_t my_opensocketfunc(void *clientp,
curlsocktype purpose,
struct curl_sockaddr *address){

return sock=socket(address->family, address->socktype, address->protocol);
}

The Request Header

The request header is defined in section 4.1 Client Requirements of RFC 6455.

The Response Header

The response header is defined in section 4.2.1. Reading the Client’s Opening Handshake of RFC 6455.

The response header fields should be checked by the header function. The format of a field is:

title: value <CRLF>

When the last field is received, the data contained in the first argument to the header function is a buffer of lenght two bytes,: <CRLF> i.e. byte 0x0d followed by 0x0a.

The structures of responses is defined here.

Sending and Receiving Messages

The format of messages is defined in section 5.2. Base Framing Protocol of RFC 6455.

A simple example is an unmask text message. In such a message the 1st byte will contain the hexadecimal value ‘0x81’. The 1st bit denotes that it is the final fragment, and the last 4 bits denotes a text message. The next byte will contain the length if shorter than 126 bytes. If the length is a 16bit number larger than 125 the byte will hold the value 0x7E, and the following 2 bytes the 16bit length. If the message is longer than 65,535 bytes, the 2nd byte will hold the value 0x7F, and the following 8 bytes will hold the length. The rest of the bytes are the payload data.

To send a message, use the C function ‘write’, as foolows:

write(sock, buff, length);

Incoming messages will be handle by the write function, given as the 3rd parameter to the curl_easy_setopt with the option ‘CURLOPT_WRITEFUNCTION’.

Read more about libcurl here.



Browsing From The Command Line / Server With cURL

cURL is a tool used for browsing the web from the shell or command-line. It supports many internet protocols, such as HTTP, FTM, POP3, IMAP, SMTP and more. See the full list here.

With libcurl installed in your system and the C API, you can browse using a C program. You can also install the extension cURL for PHP.

Steps of a cURL Program

  1. Obtain a curl handle
    For example:

    $curl_handle = curl_init();
  2. Set options to the handle.
    Define the behavior of curl when executing the request: setting the URL, cookie location, variables passed to the server, etc.
    In PHP, you can use curl_setop to set a single option, or curl_setopt_array to pass a PHP array.
    For example:

    curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,true);

    will cause cURL to store the output in a string; the value ‘false’ will cause cURL to send it to the standard output.

  3. Execute the request.
    $out=curl_exec($curl_handle);
  4. Close the handle
    curl_close($curl_handle);

As long as the handle is open, you can repeat steps 2 and 3 as many times as you need.

Another useful cURL function is curl_getinfo. In the example below, I have used

"$httpCode = curl_getinfo($curl_handle, CURLINFO_HTTP_CODE);"

to determine if the login action was successful.

 An Example – Sharing a Message in LinkedIn

Publishing a text message in LinkedIn is simple: surf to LinkedIn, find the relevant form in the response and submit it. I’ve found the login form and the publish form using HTML Dom documents. Then populated post vars according to them, and connected to the URL given in the “action” attribute of the form. The code is run in PHP CLI in Linux (“stty -echo” is a system call that suppresses the echoing of input characters in Linux).

Step I – Surf to LinkedIn

In this step cURL will send a request to get the content in linkedin.com This is the first time, so the user is not logged in, and there are no cookies. The option CURLOPT_USERAGENT will make the server believe that the request has been sent from a real browser. The option CURLOPT_COOKIEJAR will define the file from which to store and retrieve cookies.

Following is the code:

<?php
error_reporting(E_ERROR | E_PARSE); //Suppress warnings

$curl_handle = curl_init();

// Connect to the site for the first time.
curl_setopt($curl_handle,CURLOPT_URL,"https://www.linkedin.com");
curl_setopt($curl_handle,CURLOPT_USERAGENT,'Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:35.0) Gecko/20100101 Firefox/35.0');
curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,true);
curl_setopt($curl_handle,CURLOPT_COOKIEJAR,'/tmp/cookies');
$out = curl_exec($curl_handle);
if (!$out){
  echo "Error: " . curl_error($curl_handle) . "\n";
  die();
}

Step II – Get the Login Form, Populate It, and Submit It

In this step your script will read your e-mail address and password from the standard input (“php://stdin”), then populate the login form, and submit it. Using the Firefox extension DOM Inspector, I found that the Id of the form element is ‘login’, the username (e-mail) field’s name is “session_key”, and the password field’s name is “session_password”. The script willl submit the form with the input fields of type ‘hidden’ and with the entered e-mail and password. If the login was successful, the http code returned in the header would be 302, which means the output is returned from another address.

Following is the code:

$stdin = fopen('php://stdin','r');
echo "Enter e-mail:";
$email = trim(fgets($stdin));
system("stty -echo");
echo "Enter Password:";
$pass = trim(fgets($stdin));
system("stty echo");
echo "\n";
// Get the form inputs.
$doc = new DOMDocument();
$doc->loadHTML($out);

$form = $doc->getElementById('login');
$inputElements = $form->getElementsByTagName('input');
$length = $inputElements->length;

$inputs = Array();
for ($i=0;$i<$length;$i++){
  $elem=$inputElements->item($i);
  $name = $elem->getAttribute('name');
  $value = $elem->getAttribute('value');
  $inputs[$name]=$value;
}
$inputs['session_key']=$email;
$inputs['session_password']=$pass;
$keys = array_keys($inputs);
$postvars = '';

$firstInput=true;
foreach ($keys as $key){
  if (!$firstInput)
    $postvars .= '&';
  $firstInput = false;
  $postvars .= $key . "=" . urlencode($inputs[$key]);
}
$submitUrl = $form->getAttribute('action');

curl_setopt_array($curl_handle, Array(
  CURLOPT_URL=>$submitUrl,
  CURLOPT_POST=>true,
  CURLOPT_POSTFIELDS=>$postvars
));
$out=curl_exec($curl_handle);
$httpCode = curl_getinfo($curl_handle, CURLINFO_HTTP_CODE);

if ($httpCode != 302)
  die("Error - could not connect: $httpCode\n");

Step III – Post the Silly Message

After a successful login, the relevant data is stored in the cookie jar associated with the cURL handle. This time the script will read the content of the home page with the user logged in. A logged-in user can post status updates. This time, the operation is not complete until the “browser” is referred to the new address. So, we set the cURL option “CURLOPT_FOLLOWLOCATION” to true. In addition, PHP cURL allows to send an associative array as the value of the option “CURLOPT_POSTFIELDS”, a more elegant way to send POST data.

Following is the code:

// Post the message
curl_setopt($curl_handle, CURLOPT_URL, 'https://www.linkedin.com');
$out = curl_exec($curl_handle);
$doc = new DOMDocument();
$doc->loadHTML($out);
$form=$doc->getElementById('share-form');
$inputElements = $form->getElementsByTagName('input');
$length = $inputElements->length;
$inputs=Array();
for ($i=0;$i<$length;$i++){
  $elem=$inputElements->item($i);
  $name = $elem->getAttribute('name');
  $value = $elem->getAttribute('value');
  $inputs[$name]=$value;
}
$inputs['postText']="Hello! I am a message sent by a PHP script.";
$inputs['postVisibility2']='EVERYONE';
$keys=array_keys($inputs);


$formAction = $form->getAttribute('action');
if (substr($formAction,0,5)!='http:')
  $formAction = 'http://www.linkedin.com' . $formAction;

curl_setopt_array($curl_handle, Array(
  CURLOPT_URL=>$formAction,
  CURLOPT_POST=>true,
  CURLOPT_FOLLOWLOCATION=>true,
  CURLOPT_POSTFIELDS=>$inputs
));
$out = curl_exec($curl_handle);

curl_close($curl_handle);
?>