Home > Programming

Programming

Codehead Search Engine API; how to write your own Search Application in ~150 lines of code

We launched Codehead Web Services a while ago and have introduced 2 APIs so far [A Website Thumnail API and a Search Engine API].

Our goal is to provide some cool functionality that is rather hard to implement through a set of simple web services/APIs so users can create and enhance their own applications.

Here is a full working search engine using the Codehead Search Engine API:

<?php
 
	error_reporting(E_ALL);
 
	/* Save sort order and match mode prefrences in a session variable */
	session_start();
 
	/* Set default values for sort, match_mode and search_section */
	if (!isset($_SESSION['sort']))
		$_SESSION['sort'] = 'relevance_date';
 
	if (!isset($_SESSION['match_mode']))
		$_SESSION['match_mode'] = 'any';
 
	if (!isset($_SESSION['search_section']))
		$_SESSION['search_section'] = 'articles';
 
	include 'libs/codehead_api/codehead_api.php';
	$ch = new Codehead_API();
 
	/* Assume there is a $_GET['q'], @ makes sure PHP won't trow E_NOTICE and if there is no $_GET['q'] then $q will be empty or '' */
	$q = trim(@$_GET['q']);
 
	/* $rpp is results per page */
	$rpp = 25;
 
	/* $start is where to start the next page; at what item number? 10, 20, 50? This usually comes from a pager */
	$start = intval(@$_GET['start']);
 
	/* Whether or not the user want to change sort, match_mode or search section */
	if (isset($_GET['sort']))
		$_SESSION['sort'] = $_GET['sort'];
 
	if (isset($_GET['match_mode']))
		$_SESSION['match_mode'] = $_GET['match_mode'];
 
	if (isset($_GET['search_section']))
		$_SESSION['search_section'] = $_GET['search_section'];
 
?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Search - Powered By Codehead Search API</title>
 
	<style type="text/css">
		input { vertical-align: middle; }
	</style>
 
</head>
 
<body>
 
<!-- The search form -->
<form method="get" action="">
Search For: <input type="text" name="q" value="<?php echo __escape($q); ?>" /> <input type="submit" value="  GO  " />
</form>
Search in:
	<a href="?q=<?php echo __escape($q); ?>&search_section=articles">Articles</a> -
   <a href="?q=<?php echo __escape($q); ?>&search_section=forums">Forums</a> -
   <a href="?q=<?php echo __escape($q); ?>&search_section=jobs">Jobs</a> -
   <a href="?q=<?php echo __escape($q); ?>&search_section=news">News</a> -
   <a href="?q=<?php echo __escape($q); ?>&search_section=blogs">Blogs</a> - It's '<?php echo $_SESSION['search_section']; ?>' right now
<br />
Sort by:
	<a href="?q=<?php echo __escape($q); ?>&sort=relevance_date">Relevance Date</a> -
   <a href="?q=<?php echo __escape($q); ?>&sort=relevance">Relevance Only</a> -
   <a href="?q=<?php echo __escape($q); ?>&sort=date">Date Only</a> - It's '<?php echo $_SESSION['sort']; ?>' right now
<br />
Matching Mode:
	<a href="?q=<?php echo __escape($q); ?>&match_mode=any">Any Keyword</a> -
   <a href="?q=<?php echo __escape($q); ?>&match_mode=all">All Keywords</a> -
   <a href="?q=<?php echo __escape($q); ?>&match_mode=exact">Exact Match</a> - It's '<?php echo $_SESSION['match_mode']; ?>' right now
<br /><br />
 
<?php
 
	/* If $q is not empty, do the search */
	if ($q != '') {
 
		/* Do the actual search, note how simple and painless it is! */
		$results = $ch->search->do_search($_SESSION['search_section'], $q, $start, $rpp, $_SESSION['sort'], $_SESSION['match_mode']);
 
		/* Error checking is always necessary! */
		if ($ch->has_error) {
 
			echo 'OOPS! there was a temporary error.';
			/* If you want to display the internal error, go ahead and do */
 
		} else {
 
			/* All good, loop through and display the results */
			// echo '<pre>'; print_r($results); exit;
 
			if ($results['total'] > 0) {
 
				echo "Results $start to " .($start + $rpp) ." out of around {$results['total']} total results; search took {$results['search_time']} second(s)!<br /><br />";
 
				/* Show the previous/next page link */
				if ($results['total'] > $rpp && ($start + $rpp) < $results['total'])
					echo "<a href=\"?q=$q&start=" .($start + $rpp) ."\">Next $rpp results</a>";
 
				if ($start >= $rpp)
					echo " - <a href=\"?q=$q&start=" .($start - $rpp) ."\">Previous $rpp results</a><br /><br />";
				else
					echo "<br /><br />";
 
				foreach ($results['results'] as $id => $result) {
					/* Very rarely, some items might be deleted from the index so this check is necessary */
					if (!isset($result['title']))
						continue;
				?>
					<a href="<?php echo __escape($result['url']); ?>"><?php echo __escape($result['title']); ?></a><br />
					<small>From: <?php echo __escape($result['source_title']); ?> <?php echo __escape($result['formated_date']); ?></small><br />
					<?php echo __escape($result['content']); ?><br />
				<?php
					/* Codehead Search groups results for news stories */
					if (isset($result['grouped'])) {
						foreach ($result['grouped'] as $gresult) {
						?>
							<small><a href="<?php echo __escape($gresult['url']); ?>"><?php echo __escape($gresult['title']); ?></a> - <span style="color: #999999;"><?php echo __escape($gresult['source_title']); ?></span></small><br />
                  <?php
						}
					}
					echo '<br />';
				}
 
				/* Show the previous/next page link */
				if ($results['total'] > $rpp && ($start + $rpp) < $results['total'])
					echo "<a href=\"?q=$q&start=" .($start + $rpp) ."\">Next $rpp results</a>";
 
				if ($start >= $rpp)
					echo " - <a href=\"?q=$q&start=" .($start - $rpp) ."\">Previous $rpp results</a>";
 
			} else {
				echo 'Your search didn\'t match any documents.';
			}
 
		}
 
	}
 
?>
 
</body>
</html>
 
<?php
 
	function __escape($str) {
		return htmlentities($str, ENT_QUOTES, 'utf-8');
	}
 
?>

It’s very easy to understand this code but I’ll try to explain a few things about how it works.

Here is the full documentation of our Search Engine API:

Simple Usage

Here is the simplest way of searching our database for the term ‘PHP Sessions’:

<?php
 
/* Include and instantiate Codehead_API here, if you don't know how, read from the beginning of this documentation */
 
$results = $ch->search->do_search('articles', 'PHP Sessions');
 
if (!$ch->has_error) {
	if ($results['total'] > 0) {
 
		foreach ($results['results'] as $id => $result) {
			/* Very rarely, some items might be deleted from the index so this check is necessary */
			if (!isset($result['title']))
				continue;
		?>
			<a href="<?php echo __escape($result['url']); ?>"><?php echo __escape($result['title']); ?></a><br />
			<small>From: <?php echo __escape($result['source_title']); ?> <?php echo __escape($result['formated_date']); ?></small><br />
			<?php echo __escape($result['content']); ?><br />
		<?php
			/* Codehead Search groups results for news stories */
			if (isset($result['grouped'])) {
				foreach ($result['grouped'] as $gresult) {
				?>
					<small><a href="<?php echo __escape($gresult['url']); ?>"><?php echo __escape($gresult['title']); ?></a> - <span style="color: #999999;"><?php echo __escape($gresult['source_title']); ?></span></small><br />
				<?php
				}
			}
			echo '<br />';
		}
 
	} else {
		echo 'Your search didn\'t match any documents.';
	}
}
 
?>
<br />
			The function __escape() is not necessary but we like to escape EVERYTHING before printing, this function is very simple:<br /><br />
<pre lang="php">
<?php
 
function __escape($str) {
	return htmlentities($str, ENT_QUOTES, 'utf-8');
}
 
?>

Methods

There are 6 methods that you can use to retrieve search results:

<?php
 
array do_search(string $section, string $search_term [, int $offset [, int $results_per_page [, string $sort_mode [, string $match_mode]]]] )
 
array articles(string $search_term [, int $offset [, int $results_per_page [, string $sort_mode [, string $match_mode]]]] )
array jobs(string $search_term [, int $offset [, int $results_per_page [, string $sort_mode [, string $match_mode]]]] )
array forums(string $search_term [, int $offset [, int $results_per_page [, string $sort_mode [, string $match_mode]]]] )
array blogs(string $search_term [, int $offset [, int $results_per_page [, string $sort_mode [, string $match_mode]]]] )
array news(string $search_term [, int $offset [, int $results_per_page [, string $sort_mode [, string $match_mode]]]] )
 
?>

Below are detailed descriptions of these arguments and the reply from the server…

Search Sections

Codehead Search Engine API has 5 sections for searching:

1 – forums
2 – articles
3 – blogs
4 – news
5 – jobs

You could search these sections directly like:
```
<?php
 
$results = $ch->search->jobs('Python');
 
?>
```
Or use the generic do_search method like:
```
<?php
 
$results = $ch->search->do_search('jobs', 'Java');
 
?>
```
!!!Please note that, the default section is ‘articles’ so if you misspell the section or pass in an empty section, the API will respond with all the articles matching your term!!!
Offset and Results Per Page (RPP)

The offset is the start offset in which you want the results; if you have a pager and a user clicks on ‘Page 2′ then you must pass in the starting offset of the page 2. The default value for offset is 0 which means starting at the beginning of the result set.

Results per page (RPP) is the number of search results that you want to show on each of your SERP pages. Valid values for RPP are 10 or 25 and the default value is 10.

As an example, suppose you are showing 10 search results per page so the starting offset of your page 3 would be 3 * 10 or 30 so:
```
<?php
 
$results = $ch->search->do_search('news', 'bing', 30, 10);
 
?>
```
Will display the page 3.
Sort Mode

This is the mode in which the Search Engine API will sort your search results and valid values are:

relevance_date

Which will sort your results in order of relevance to the search term and also their freshness, this is our prefered mode.

relevance – Also Default

Will only sort your results based on their relevance to the search term. This is the default value for $sort_mode.

date

Will only sort your results based on date and it’s the least favorite sorting mode.

Examples
```
<?php
 
$results = $ch->search->do_search('news', 'bing', 30, 10, 'relevance_date');
/* or */
$results = $ch->search->do_search('news', 'bing', 30, 10, 'relevance');
/* or */
$results = $ch->search->do_search('news', 'bing', 30, 10, 'date');
 
?>
```
Match Mode

There are 3 matching modes:

any

This will tell the Search Engine API to match any of the keywords in your search term, for example if your search term is ‘SEO Jobs’, the API will return all the documents with either ‘SEO’ or ‘Jobs’ in them and that will include documents with both keywords in them too.

all – Also Default

Will return all the documents with all the keywords in them. This is also the default $match_mode.

exact

This will match all the documents with all the keywords in them but this time it will make sure that the keywords will appear in the exact order in the search term. This is basically phrase match, for example if your search term is ‘PHP Jobs Las Vegas’, the Search Engine API will return all the documents with the exact phrase ‘PHP Jobs Las Vegas’ in them.

Examples
```
<?php
 
$results = $ch->search->do_search('news', 'bing', 30, 10, 'relevance_date', 'any');
/* or */
$results = $ch->search->do_search('news', 'bing', 30, 10, 'relevance_date', 'all');
/* or */
$results = $ch->search->do_search('news', 'bing', 30, 10, 'relevance_date', 'exact');
 
?>
```
All About The Results Array

The return value of any of the above methods is an array that contains the search results along with some data about the search.

The components of this array are:

1 – results – Which is an array containing the search results.
2 – total – Is the total number of matches, this is useful when building pagers.
3 – encoding – Is the encoding of the items in the result.
4 – time_now – The current time of the server in Unix timestamp.
5 – start – The start offset of the results.
6 – rpp – Results per page.
7 – sort – Sort mode.
8 – match_mode – Match mode.
9 – search_time – The time it took to perform the search in milliseconds.

!!!Please note that, each item in the ‘results’ might contain another sub-array named ‘grouped’ which are the results that were similar to that item; this will only be the case when you search the news!!!

The best way to examine the ‘results’ array is to perform a search and then print_r() the results.
```
<?php
 
$results = $ch->search->do_search('blogs', 'snow leopard', 30, 10, 'relevance_date', 'any');
print_r($results);
 
?>
```

Any problems with the API? You can create a support ticket in your Codehead Webservice Account under Account > Help.

Good Luck

P.S. Check out the original Codehead Search, our super-fast, custom search engine, where you can scour the best content on the net for computer and web related jobs, forum posts, articles and tutorials, blog posts and news stories all in one place.

Comments (0) Filed under: Codehead Web Services,PHP,Programming,Search Engines,Web Development Posted by: Codehead

How to write a permission system using bits and bitwise operations in PHP

I wrote this in PHP but you can use the same concept in other languages, I also assume an understanding of bits, bytes, binary to decimal conversion and vice-versa and bitwise operations on numbers like ‘or’, ‘and’ and ‘xor’ etc. if you have no idea, search and read about these first. You don’t have to be a guru but you should have an idea. Here are some pages to get you started:

http://en.wikipedia.org/wiki/Byte
http://en.wikipedia.org/wiki/Bitwise_operation
http://us.php.net/manual/en/language.operators.bitwise.php
Some binary to decimal calculators to make it easier

We will use simple numbers to represent different permissions and as you might know a number is a collection of bytes. For example: an integer is usually 4 bytes. Although you don’t have to worry about the size of a number in a high level language like PHP but a little understanding of representation of numbers will help you better understand this technique.

So let’s assume when I say:

<?php
 
	$user_perms = 7;
 
?>

Internally the variable $user_perms looks like this:

|0|0|0|0|0|0|0|0|0|0|0|0|0|1|1|1|

This is a 2 byte representation of number 7, although, it might not look like this internally – it looks similar. Just assume this for now.

Let’s say that your application supports 4 functions that a user can use:

1 – Post a blog post
2 – Comment on blog posts
3 – Edit posts
4 – Delete posts

Normally, you could have 4 fields in your database table (structure or whatever) for a user titled:

1 – can_post
2 – can_comment
3 – can_edit
4 – can_delete

This is not good, 4 additional fields for your user table and who knows, what if your application has 100 functions? Do you want to add 100 fields to your user table?

With bits, you can have only 1 column and track all the permissions.

1 – perms

To do this, we will have to assign numbers for each of the functions: (Tip: use one of the calculators in the above list )

1 – Post a blog post |0|0|0|0|0|0|0|1| is 1 in decimal
2 – Comment on blog posts |0|0|0|0|0|0|1|0| is 2 in decimal
3 – Edit posts |0|0|0|0|0|1|0|0| is 4 in decimal
4 – Delete posts |0|0|0|0|1|0|0|0| is 8 in decimal

So you could have an array like this:

<?php
 
	$perms = array(
		'can_post' => 1,
		'can_comment' => 2,
		'can_edit' => 4,
		'can_delete' => 8
	);
 
?>

Almost there, let’s look at user’s perms field now.

I hope you know about bitwise ‘or’, when you ‘or’ 1 and 1 you get 1; 0 ‘or’ 1 is 1; 1 ‘or’ 0, is 1 and finally 0 ‘or’ 0 is 0, it’s just like the meaning of ‘or’ in the English language.

Similarly, bitwise ‘and’; when you ‘and’ 1 and 1 you get 1; 0 ‘and’ 1 is 0; 1 ‘and’ 0, is 0 and finally 0 ‘and’ 0 is 0, again it’s just like the meaning of ‘or’ in the English language.

Bitwise ‘xor’; when you ‘xor’ 1 and 1 you get 0; 0 ‘xor’ 1 is 1; 1 ‘xor’ 0, is 1 and finally 0 ‘xor’ 0 is 0.

So suppose you want to give a user permissions to post a blog post, post a comment and edit posts but not delete posts, you do it like this:

<?php
 
	$user_perms = $perms['can_post'] | $perms['can_comment'] | $perms['can_edit'];
 
?>

Note that, in PHP ‘|’ means ‘or’, so what just happened is something like this:

|0|0|0|0|0|0|0|1| ‘or’
|0|0|0|0|0|0|1|0| ‘or’
|0|0|0|0|0|1|0|0|
_______________________
|0|0|0|0|0|1|1|1|

Now $user_perms has the value 7 and |0|0|0|0|0|1|1|1| in it internally.

Suppose that this is on top of your post_blog.php or where ever you want to handle permissions for posting a blog, the only thing you need to do is:

<?php
 
	if ($user_perms & $perms['can_post']) {
		/* He/She has permissios to do this */
	} else {
		/* He/She doesn't */
	}
 
?>

In PHP ‘&’ is for bitwise ‘and’, please also note that ‘&&’ is logical ‘and’ and doesn’t operate on individual bits.

This is exactly what just happened:

|0|0|0|0|0|1|1|1| ‘and’
|0|0|0|0|0|0|0|1|
_______________________
|0|0|0|0|0|0|0|1|

So that’s ‘one’ not ’0′, which means ‘if’ passes and the user has permissions to do this. But when it comes to deleting posts:

<?php
 
	if ($user_perms & $perms['can_delete']) {
		/* He/She does permissios to do this */
	} else {
		/* He/She doesn't */
	}
 
?>

Thus:

|0|0|0|0|0|1|1|1| ‘and’
|0|0|0|0|1|0|0|0|
_______________________
|0|0|0|0|0|0|0|0|

It’s ‘zero’ so ‘if’ fails and you show an error message or whatever it is you do.

To add ‘delete’ permissions, you use ‘or’ again:

<?php
 
	$user_perms |= $perms['can_delete'];
 
?>

So this happens:

|0|0|0|0|0|1|1|1| ‘or’
|0|0|0|0|1|0|0|0|
_______________________
|0|0|0|0|1|1|1|1|

To take away permissions you use ‘xor’:

<?php
 
	$user_perms ^= $perms['can_delete'];
 
?>

And this will happen:

|0|0|0|0|1|1|1|1| ‘xor’
|0|0|0|0|1|0|0|0|
_______________________
|0|0|0|0|0|1|1|1|

And delete permissions are gone!

Now let’s take away post permissions:

<?php
 
	$user_perms ^= $perms['can_post'];
 
?>

Thus:

|0|0|0|0|0|1|1|1| ‘xor’
|0|0|0|0|0|0|0|1|
_______________________
|0|0|0|0|0|1|1|0|

So this was just the basics, you can build on this and do more once you understand.

I hope this post will help someone

Comments (10) Filed under: PHP,Programming,Security,Web Development Posted by: Codehead

Implementation of a Vector data structure in C

I use Python a lot but it’s very slow in some cases which I then use C because it can be 1,000s X faster than Python.

Here is a Vector that I just wrote to use in my new project:

vector.h

/**
 * Hamid Alipour
 */
 
#ifndef __VECTORH__
#define __VECTORH__
 
#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <string.h>
#include <assert.h>
 
#define VECTOR_INIT_SIZE    4
#define VECTOR_HASSPACE(v)  (((v)->num_elems + 1) <= (v)->num_alloc_elems)
#define VECTOR_INBOUNDS(i)	(((int) i) >= 0 && (i) < (v)->num_elems)
#define VECTOR_INDEX(i)		((char *) (v)->elems + ((v)->elem_size * (i)))
 
typedef struct _vector {
	void *elems;
	size_t elem_size;
	size_t num_elems;
	size_t num_alloc_elems;
    void (*free_func)(void *);
} vector;
 
extern void vector_init(vector *, size_t, size_t, void (*free_func)(void *));
extern void vector_dispose(vector *);
extern void vector_copy(vector *, vector *);
extern void vector_insert(vector *, void *, size_t index);
extern void vector_insert_at(vector *, void *, size_t index);
extern void vector_push(vector *, void *);
extern void vector_pop(vector *, void *);
extern void vector_shift(vector *, void *);
extern void vector_unshift(vector *, void *);
extern void vector_get(vector *, size_t, void *);
extern void vector_remove(vector *, size_t);
extern void vector_transpose(vector *, size_t, size_t);
extern size_t vector_length(vector *);
extern size_t vector_size(vector *);
extern void vector_get_all(vector *, void *);
extern void vector_cmp_all(vector *, void *, int (*cmp_func)(const void *, const void *));
extern void vector_qsort(vector *, int (*cmp_func)(const void *, const void *));
static void vector_grow(vector *, size_t);
static void vector_swap(void *, void *, size_t);
 
#endif

And vector.c

/**
 * Hamid Alipour
 */
 
#include "vector.h"
 
extern void vector_init(vector *v, size_t elem_size, size_t init_size, void (*free_func)(void *))
{
	v->elem_size = elem_size;
	v->num_alloc_elems = (int) init_size > 0 ? init_size : VECTOR_INIT_SIZE;
	v->num_elems = 0;
	v->elems = malloc(elem_size * v->num_alloc_elems);
	assert(v->elems != NULL);
	v->free_func = free_func != NULL ? free_func : NULL;
}
 
extern void vector_dispose(vector *v)
{
	size_t i;
 
	if (v->free_func != NULL) {
		for (i = 0; i < v->num_elems; i++) {
			v->free_func(VECTOR_INDEX(i));
		}
	}
 
	free(v->elems);
}
 
 
extern void vector_copy(vector *v1, vector *v2)
{
	v2->num_elems = v1->num_elems;
	v2->num_alloc_elems = v1->num_alloc_elems;
	v2->elem_size = v1->elem_size;
 
	v2->elems = realloc(v2->elems, v2->num_alloc_elems * v2->elem_size);
	assert(v2->elems != NULL);
 
	memcpy(v2->elems, v1->elems, v2->num_elems * v2->elem_size);
}
 
extern void vector_insert(vector *v, void *elem, size_t index)
{
	void *target;
 
	if ((int) index > -1) {
		if (!VECTOR_INBOUNDS(index))
			return;
		target = VECTOR_INDEX(index);
	} else {
		if (!VECTOR_HASSPACE(v))
			vector_grow(v, 0);
		target = VECTOR_INDEX(v->num_elems);
		v->num_elems++; /* Only grow when adding a new item not when inserting in a spec indx */
	}
 
	memcpy(target, elem, v->elem_size);
}
 
extern void vector_insert_at(vector *v, void *elem, size_t index)
{
	if ((int) index < 0)
		return;
 
	if (!VECTOR_HASSPACE(v))
		vector_grow(v, 0);
 
	if (index < v->num_elems)
		memmove(VECTOR_INDEX(index + 1), VECTOR_INDEX(index), (v->num_elems - index) * v->elem_size);
 
	/* 1: we are passing index so insert won't increment this 2: insert checks INBONDS... */
	v->num_elems++;
 
	vector_insert(v, elem, index);
}
 
extern void vector_push(vector *v, void *elem)
{
	vector_insert(v, elem, -1);
}
 
extern void vector_pop(vector *v, void *elem)
{
	memcpy(elem, VECTOR_INDEX(v->num_elems - 1), v->elem_size);
	v->num_elems--;
}
 
extern void vector_shift(vector *v, void *elem)
{
	memcpy(elem, v->elems, v->elem_size);
	memmove(VECTOR_INDEX(0), VECTOR_INDEX(1), v->num_elems * v->elem_size);
 
	v->num_elems--;
}
 
extern void vector_unshift(vector *v, void *elem)
{
	if (!VECTOR_HASSPACE(v))
		vector_grow(v, v->num_elems + 1);
 
	memmove(VECTOR_INDEX(1), v->elems, v->num_elems * v->elem_size);
	memcpy(v->elems, elem, v->elem_size);
 
	v->num_elems++;
}
 
extern void vector_transpose(vector *v, size_t index1, size_t index2)
{
	vector_swap(VECTOR_INDEX(index1), VECTOR_INDEX(index2), v->elem_size);
}
 
static void vector_grow(vector *v, size_t size)
{
	if (size > v->num_alloc_elems)
		v->num_alloc_elems = size;
	else
		v->num_alloc_elems *= 2;
 
	v->elems = realloc(v->elems, v->elem_size * v->num_alloc_elems);
	assert(v->elems != NULL);
}
 
extern void vector_get(vector *v, size_t index, void *elem)
{
	assert((int) index >= 0);
 
	if (!VECTOR_INBOUNDS(index)) {
		elem = NULL;
		return;
	}
 
	memcpy(elem, VECTOR_INDEX(index), v->elem_size);
}
 
extern void vector_get_all(vector *v, void *elems)
{
	memcpy(elems, v->elems, v->num_elems * v->elem_size);
}
 
extern void vector_remove(vector *v, size_t index)
{
	assert((int) index > 0);
 
	if (!VECTOR_INBOUNDS(index))
		return;
 
	memmove(VECTOR_INDEX(index), VECTOR_INDEX(index + 1), v->elem_size);
	v->num_elems--;
}
 
extern void vector_remove_all(vector *v)
{
	v->num_elems = 0;
	v->elems = realloc(v->elems, v->num_alloc_elems);
	assert(v->elems != NULL);
}
 
extern size_t vector_length(vector *v)
{
	return v->num_elems;
}
 
extern size_t vector_size(vector *v)
{
	return v->num_elems * v->elem_size;
}
 
extern void vector_cmp_all(vector *v, void *elem, int (*cmp_func)(const void *, const void *))
{
	size_t i;
	void *best_match = VECTOR_INDEX(0);
 
	for (i = 1; i < v->num_elems; i++)
		if (cmp_func(VECTOR_INDEX(i), best_match) > 0)
			best_match = VECTOR_INDEX(i);
 
	memcpy(elem, best_match, v->elem_size);
}
 
extern void vector_qsort(vector *v, int (*cmp_func)(const void *, const void *))
{
	qsort(v->elems, v->num_elems, v->elem_size, cmp_func);
}
 
static void vector_swap(void *elemp1, void *elemp2, size_t elem_size)
{
	void *tmp = malloc(elem_size);
 
	memcpy(tmp, elemp1, elem_size);
	memcpy(elemp1, elemp2, elem_size);
	memcpy(elemp2, tmp, elem_size);
 
             free(tmp); /* Thanks to gromit */
}

It works I will write another post with some example usage soon…

Notice that, all the functions that return a value, return it through their arguments and not the “return” statement, this is because I wanted the client to be responsible for allocating/freeing memory for these variables and that makes everyone’s job easier.

As always, I’m open to criticism/ideas and please use it at your own risk.

Comments (9) Filed under: C Programming,Data Structures,Low Level,Programming Posted by: Codehead

Basic threading in JavaScript; JavaScript Worker Threads

I was searching the web yesterday for this and didn’t really find a simple way of doing this and suddenly, I remembered something.

It’s extremely simple, someone could build on it with all sorts of features but for now, here is an example:
http://images.code-head.com/code/javascript/js-threads.html

As you can see there are two counters, one is counting up and the other down simultaneously.

Here is the start_thread function:

function thread_start(callback) {
   setTimeout(callback, 1);
   return true;
}

The trick is that setTimeout *does not* block the execution and starts another thread to run the callback function

I hope this helps someone

Comments (5) Filed under: Fun,General,JavaScript,jQuery,Programming Posted by: Codehead

Missing from Python: kill-able threads and popens with timeouts…

It would be nice to be able to kill threads anytime you wanted, there are solutions to this but are not part of the Python library and I didn’t yet get a chance to try them… Actually there is only one promising solution to this.

The other thing I would love to have in Python is to be able to call popen with a second parameter which is a timeout, That is (obviously) popen would give up and return if the call was taking more than x number of seconds…

Comments (0) Filed under: Programming,Python Posted by: Codehead

Python win32api and pywintypes.error: (5, ‘TerminateProcess’, ‘Access is denied.’)

I was working on a piece of code that was supposed to close a bunch of processes ‘with the same name’ right after each other and was getting this error message:
pywintypes.error: (5, ‘TerminateProcess’, ‘Access is denied.’)

The problem was that I was doing this in a loop and Windows didn’t get enough time to close the first process yet. To fix this, just add a time.sleep(0.5) in your loop and you will be all set…

I hope this helps

Here is the code I’m using (found in Python mailing list):

import time
import win32api, win32pdhutil, win32con
import win32pdh, string
 
# ***********************************************************************
# ***********************************************************************
def GetAllProcesses():
  object = "Process"
  items, instances = win32pdh.EnumObjectItems(None,None,object,
win32pdh.PERF_DETAIL_WIZARD)
  return instances
# ***********************************************************************
 
 
# ***********************************************************************
# ***********************************************************************
def GetProcessID ( name ) :
  object = "Process"
  items, instances = win32pdh.EnumObjectItems(None,None,object, win32pdh.PERF_DETAIL_WIZARD)
 
  val = None
  if name in instances :
    hq = win32pdh.OpenQuery()
    hcs = []
    item = "ID Process"
    path = win32pdh.MakeCounterPath( (None,object,name, None, 0, item) )
    hcs.append(win32pdh.AddCounter(hq, path))
    win32pdh.CollectQueryData(hq)
    time.sleep(0.01)
    win32pdh.CollectQueryData(hq)
 
    for hc in hcs:
      type, val = win32pdh.GetFormattedCounterValue(hc, win32pdh.PDH_FMT_LONG)
      win32pdh.RemoveCounter(hc)
    win32pdh.CloseQuery(hq)
    return val
# ***********************************************************************
 
# ***********************************************************************
# ***********************************************************************
def Kill_Process_pid ( pid ) :
  handle = win32api.OpenProcess(win32con.PROCESS_TERMINATE, 0, pid) #get process handle
  win32api.TerminateProcess(handle, -1) #kill by handle
  win32api.CloseHandle(handle)        #close api
# ***********************************************************************
 
 
# ***********************************************************************
# ***********************************************************************
def Kill_Process ( name ) :
  pid = GetProcessID(name)
  if pid:
    try:
        Kill_Process_pid(pid)
        return True
    except:
        pass
  else:
    return False
# ***********************************************************************

I just modified the last function, it’s kind of funny I know but that works for me, then I call it like:

print 'Killing IEs...',
while Kill_Process('iexplore'):
   time.sleep(0.5)
print 'Done!'

Comments (1) Filed under: Programming,Python Posted by: Codehead

Ungzip a string in Python – gzinflate in Python

I was very surprised that I couldn’t do this easily in Python, so here is the solution to this:

import zlib
# ...
ungziped_str = zlib.decompressobj().decompress('x\x9c' + gziped_str)

Wow, PHP does it like this:

$ungziped_str = gzinflate($gziped_str);

Comments (1) Filed under: Programming,Python Posted by: Codehead

Python “Unhandled exception in thread started by Error in sys.excepthook:”

I was getting this error message and I was using a Queue object to queue some jobs and block everything until all threads are done with:

the_queue.join()

Well, in my particular case, I was getting these error messages the work was not done in worker threads:

Unhandled exception in thread started by
Error in sys.excepthook:

Original exception was:

So what I did was this:

threads = []
for i in range(0, max_threads):
   thread = Worker()
   thread.start()
   threads.append(thread)
 
# And then...
for thread in threads:
   thread.join()

And this fixed my issue, this will make sense to people with this problem

Comments (0) Filed under: Programming,Python Posted by: Codehead

PML – A Python template engine

I finally decided to release PML as an open source software. I developed this template engine for a project that unfortunately never happened using Python.

Here is a sample application using PML and Yahoo! Search API:
http://web-search.code-head.com/

You can download it here:

https://github.com/Codingrecipes/PML

The web search example is also included in the download.

PML was written to be fast, simple and compact. Here are some of it’s features:
1 – Template filters
2 – Output filters
3 – Template variable filters
4 – Template cache – default
5 – Bytecode cache – default
6 – Complete output buffer cache
7 – Garbage collection
8 – Output compression – GZIP
9 – A powerful, quick compiler
10 – Ability to add helpers easily
11 – Ability to add custom compiler tags
12 – Auto escaping your variables, even lists, dicts, and tuples
13 – Compile templates once until you edit them

And more.

The web search application above is included in your download and it demonstrates almost all the aspects of this powerful template engine.

Comments (2) Filed under: Programming,Python,Web Design,Web Development Posted by: Codehead

A PHP Competition 2

Say that you are generating an XML sitemap for your site and there are 5,000 URLs.

Here is the code:

<?php
 
	$buffer =
'<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
';
 
	$results = fetch_5000_urls_order_by_importance();
	$count = 0;
	while ($row = $results->fetch_next_url()) {
		$priority = ##########################################;
		$url = $row['url'];
		$date = date('Y-m-d', $row['date']);
		$buffer .= "
			<url>
				<loc>$url</loc>
				<lastmod>$date</lastmod>
				<changefreq>weekly</changefreq>
				<priority>$priority</priority>
			</url>
			";
	}
 
	$buffer .= '
	</urlset>
	';
 
	echo $buffer;
 
?>

What you want to do is to give each URL a priority; URLs between 0 to 1000 will have a priority of 0.9
1000 to 2000 => 0.8
2000 to 3000 => 0.7
3000 to 5000 => 0.6
4000 to 5000 => 0.5

Replace ########################################## with the code that calculates the priorities, and remember not to use ; in it

Comments (9) Filed under: Fun,PHP,Programming Posted by: Codehead

« Newer Posts — Older Posts »

Back To Previous Page

Programming

Codehead Search Engine API; how to write your own Search Application in ~150 lines of code

Simple Usage

Methods

Search Sections

Offset and Results Per Page (RPP)

Sort Mode

relevance_date

relevance – Also Default

date

Examples

Match Mode

any

all – Also Default

exact

Examples

All About The Results Array

How to write a permission system using bits and bitwise operations in PHP

Implementation of a Vector data structure in C

Basic threading in JavaScript; JavaScript Worker Threads

Missing from Python: kill-able threads and popens with timeouts…

Python win32api and pywintypes.error: (5, ‘TerminateProcess’, ‘Access is denied.’)

Ungzip a string in Python – gzinflate in Python

Python “Unhandled exception in thread started by Error in sys.excepthook:”

PML – A Python template engine

A PHP Competition 2

CODEHEAD

Downloads

Categories

RSS Feed

Search

Programming

Simple Usage

Methods

Search Sections

Offset and Results Per Page (RPP)

Sort Mode

relevance_date

relevance – Also Default

date

Examples

Match Mode

any

all – Also Default

exact

Examples

All About The Results Array

CODEHEAD

Downloads

Categories

RSS Feed

Search

Tags