HTTP Browsing

Introduction

This article is about demonstrating how to simulate a browser request to any web page.

Can be used:

1. When we want to get the content of a web page like when developing a web crawler.

2. When we want a certain service which will accept parameters and return information in return.

The code below should be used with ASP pages.

The ASP page will receive 2 form parameters using the POST method and return information in return.

Getting Started

Create a .NET Web Project called DEMO ( or anything else ).
Visual Studio will create the WebForm.aspx file automatically.

In the code behind section, under Page_Load event you can check the Request.Form collection
and us Response.Write to write back information to requesting client.

Additional explanation is documented in the code segment.

string param1 = "value1";

string param2 = "value2";

ASCIIEncoding encoding = new ASCIIEncoding();

string postData = string.Format("param1={0}&param2={1}", param1, param2 );

byte[] buffer = encoding.GetBytes( postData );

// Prepare web request...
HttpWebRequest myRequest =
(HttpWebRequest)WebRequest.Create("http://localhost/DEMO/WebForm1.aspx");

// We use POST ( we can also use GET )
myRequest.Method = "POST";

// Set the content type to a FORM
myRequest.ContentType ="application/x-www-form-urlencoded";

// Get length of content
myRequest.ContentLength = buffer.Length;

// Get request stream
Stream newStream = myRequest.GetRequestStream();

// Send the data.
newStream.Write(buffer,0,buffer.Length);

// Close stream
newStream.Close();

// Assign the response object of 'HttpWebRequest' to a 'HttpWebResponse' variable.
HttpWebResponse myHttpWebResponse= (HttpWebResponse)myRequest.GetResponse();

// Display the contents of the page to the console.
Stream streamResponse=myHttpWebResponse.GetResponseStream();

// Get stream object
StreamReader streamRead = new StreamReader( streamResponse );

Char[] readBuffer = new Char[256];

// Read from buffer
int count = streamRead.Read( readBuffer, 0, 256 );

while (count > 0)
{
     // get string
     String resultData = new String( readBuffer, 0, count);

     // Write the data
     Console.WriteLine( resultData );

     // Read from buffer
     count = streamRead.Read( readBuffer, 0, 256);

}

// Release the response object resources.
streamRead.Close();

streamResponse.Close();

// Close response
myHttpWebResponse.Close();