You Are Here Home > Python

Python

Python SQLite3 In Multiple Threads

If you create your database in a thread – usually the main thread – and try to use it in another thread your will get:

ProgrammingError: SQLite objects created in a thread can only be used in that
same thread.The object was created in thread id SOME_ID and this is thread
id SOME_ID

This is because you created the ‘cursor’ object in the main thread, if you for example create only the connection object in the main thread and create ‘cursor’s when you want to query the database, your problem will be solved.

Of course you must take care of the concurrency issues that you might have but other than that, you could do something like this:

import sqlite3
 
class datastore:
 
    def __init__(self):
        self.data_file = 'path_to_db_file'
 
    def connect(self):
        self.conn = sqlite3.connect(self.data_file)
        return self.conn.cursor()
 
    def disconnect(self):
        self.cursor.close()
 
    def free(self, cursor):
        cursor.close()
 
    def write(self, query, values = ''):
        cursor = self.connect()
        if values != '':
            cursor.execute(query, values)
        else:
            cursor.execute(query)
        self.conn.commit()
        return cursor
 
    def read(self, query, values = ''):
        cursor = self.connect()
        if values != '':
            cursor.execute(query, values)
        else:
            cursor.execute(query)
        return cursor

This will fix the issue, it creates cursors for each query and returns it to the caller, the only thing is that you must ‘close’ the cursor when you are done with it. Here I have a member function ‘free’. (probably because of my PHP brain damage)

I hope this helps…

Python SQLite3 In Multiple Threads
Comments (0)   Filed under: Programming, Python, SQLite   Posted by: Codehead

The Best Python Code Editor: Komodo Edit

I looked for a Python editor a lot, I found Pydev which is a plugin for Eclipse and since Eclipse sucks, it’s ugly and the code looks ugly too, it just kills my creativity, it’s also slow on top of that.

Then I found IronPython plugin for Microsoft Visual Studio, this one requires Visual Studio which is paid and it doesn’t make your life easier, for example if I have:

def some_func():
   pass

Now, after “pass” if I hit “Enter” I want to get back to the beginning of the next line but this wasn’t happening in IronPython. (+ a bunch of other things)

I guess I got spoiled because I use Dreamweaver to write PHP and it just does everything as you expect, it’s smooth and well thought out, much like other Macromedia products; take Fireworks for example, those who use Fireworks and Photoshop know the brilliance behind the design of Fireworks’s UI… Why didn’t Macromedia buy Adobe?!

Anyway, if you are like me and feel the same way, try Komodo Edit. I have nothing to do with it or ActiveState but I have to say that this editor made my life so much easier and I’m not looking back. It also has support for a bunch of other languages like PHP etc.

EDIT: So after using this great text editor for a little while, I decided to write a quick list of pros and cons:

Pros:
1 – It’s free.
2 – It’s open-source.
3 – It’s nice looking. (I care about this, I’m convinced that it effects creativity…)
4 – It’s smooth and fast.
5 – It does what you expect it to do; very intuitive.
6 – It’s written by people who love writing code.
7 – It’s cross-platform.
8 – It supports: PHP, Python, Ruby, Perl and Tcl, plus JavaScript, CSS, HTML and template languages like RHTML, Template-Toolkit, HTML-Smarty and Django.
9 – It supports code completion.
10 – It has great help and docs.

Cons:
None.

Happy Coding…

The Best Python Code Editor: Komodo Edit
Comments (5)   Filed under: Annoying Stuff, IDEs, PHP, Programming, Python, Web Development   Posted by: Codehead

Check To See If a File Exists

import os
 
if os.path.exists(path_to_file):
   # Do stuff
Check To See If a File Exists
Comments (0)   Filed under: Python   Posted by: Codehead

Missing from Python: kill-able threads and popens with timeouts…

It would be nice to be able to kill threads anytime you wanted, there are solutions to this but are not part of the Python library and I didn’t yet get a chance to try them… Actually there is only one promising solution to this.

The other thing I would love to have in Python is to be able to call popen with a second parameter which is a timeout, That is (obviously) popen would give up and return if the call was taking more than x number of seconds…

Missing from Python: kill-able threads and popens with timeouts…
Comments (0)   Filed under: Programming, Python   Posted by: Codehead

Python win32api and pywintypes.error: (5, ‘TerminateProcess’, ‘Access is denied.’)

I was working on a piece of code that was supposed to close a bunch of processes ‘with the same name’ right after each other and was getting this error message:
pywintypes.error: (5, ‘TerminateProcess’, ‘Access is denied.’)

The problem was that I was doing this in a loop and Windows didn’t get enough time to close the first process yet. To fix this, just add a time.sleep(0.5) in your loop and you will be all set…

I hope this helps :)

Here is the code I’m using (found in Python mailing list):

import time
import win32api, win32pdhutil, win32con
import win32pdh, string
 
# ***********************************************************************
# ***********************************************************************
def GetAllProcesses():
  object = "Process"
  items, instances = win32pdh.EnumObjectItems(None,None,object,
win32pdh.PERF_DETAIL_WIZARD)
  return instances
# ***********************************************************************
 
 
# ***********************************************************************
# ***********************************************************************
def GetProcessID ( name ) :
  object = "Process"
  items, instances = win32pdh.EnumObjectItems(None,None,object, win32pdh.PERF_DETAIL_WIZARD)
 
  val = None
  if name in instances :
    hq = win32pdh.OpenQuery()
    hcs = []
    item = "ID Process"
    path = win32pdh.MakeCounterPath( (None,object,name, None, 0, item) )
    hcs.append(win32pdh.AddCounter(hq, path))
    win32pdh.CollectQueryData(hq)
    time.sleep(0.01)
    win32pdh.CollectQueryData(hq)
 
    for hc in hcs:
      type, val = win32pdh.GetFormattedCounterValue(hc, win32pdh.PDH_FMT_LONG)
      win32pdh.RemoveCounter(hc)
    win32pdh.CloseQuery(hq)
    return val
# ***********************************************************************
 
# ***********************************************************************
# ***********************************************************************
def Kill_Process_pid ( pid ) :
  handle = win32api.OpenProcess(win32con.PROCESS_TERMINATE, 0, pid) #get process handle
  win32api.TerminateProcess(handle, -1) #kill by handle
  win32api.CloseHandle(handle)        #close api
# ***********************************************************************
 
 
# ***********************************************************************
# ***********************************************************************
def Kill_Process ( name ) :
  pid = GetProcessID(name)
  if pid:
    try:
        Kill_Process_pid(pid)
        return True
    except:
        pass
  else:
    return False
# ***********************************************************************

I just modified the last function, it’s kind of funny I know but that works for me, then I call it like:

print 'Killing IEs...',
while Kill_Process('iexplore'):
   time.sleep(0.5)
print 'Done!'
Python win32api and pywintypes.error: (5, ‘TerminateProcess’, ‘Access is denied.’)
Comments (0)   Filed under: Programming, Python   Posted by: Codehead

Ungzip a string in Python – gzinflate in Python

I was very surprised that I couldn’t do this easily in Python, so here is the solution to this:

import zlib
# ...
ungziped_str = zlib.decompressobj().decompress('x\x9c' + gziped_str)

Wow, PHP does it like this:

$ungziped_str = gzinflate($gziped_str);
Ungzip a string in Python – gzinflate in Python
Comments (1)   Filed under: Programming, Python   Posted by: Codehead

Python “Unhandled exception in thread started by Error in sys.excepthook:”

I was getting this error message and I was using a Queue object to queue some jobs and block everything until all threads are done with:

the_queue.join()

Well, in my particular case, I was getting these error messages the work was not done in worker threads:

Unhandled exception in thread started by
Error in sys.excepthook:

Original exception was:

So what I did was this:

threads = []
for i in range(0, max_threads):
   thread = Worker()
   thread.start()
   threads.append(thread)
 
# And then...
for thread in threads:
   thread.join()

And this fixed my issue, this will make sense to people with this problem ;)

Python “Unhandled exception in thread started by Error in sys.excepthook:”
Comments (0)   Filed under: Programming, Python   Posted by: Codehead

PML – A Python template engine

I finally decided to release PML as an open source software. I developed this template engine for a project that unfortunately never happened using Python.

Here is a sample application using PML and Yahoo! Search API:
http://web-search.code-head.com/

You can download it here:

[D O W N L O A D]

The web search example is also included in the download.

PML was written to be fast, simple and compact. Here are some of it’s features:
1 – Template filters
2 – Output filters
3 – Template variable filters
4 – Template cache – default
5 – Bytecode cache – default
6 – Complete output buffer cache
7 – Garbage collection
8 – Output compression – GZIP
9 – A powerful, quick compiler
10 – Ability to add helpers easily
11 – Ability to add custom compiler tags
12 – Auto escaping your variables, even lists, dicts, and tuples
13 – Compile templates once until you edit them

And more.

The web search application above is included in your download and it demonstrates almost all the aspects of this powerful template engine.

PML – A Python template engine
Comments (0)   Filed under: Programming, Python, Web Design, Web Development   Posted by: Codehead

A little Program For Monitoring Your Websites, With Alarm

I hope someone will find this useful:

###
 #    Hamid Alipour
###
 
###
 # Config
#####################################################
urls = ['http://www.P U T  Y O U R  U R L S  H E R E.com/']
url_timeout = 5
sleep_time = 5 * 60
failed_count_before_alarm = 10
sleep_time_on_failure = 15
#####################################################
 
import urllib2
import winsound
import time
 
class SiteMonitor:
 
    def __init__(self):
        pass
 
    def start(self):
        while True:
            for url in urls:
                if not self.check_url(url):
                    if not self.is_it_live(url):
                        self.alarm()
            self.sleep(sleep_time)
 
    def is_it_live(self, url):
        failed_count = 0
        while failed_count < failed_count_before_alarm:
            if not self.check_url(url):
                failed_count += 1
            else:
                return True
            self.sleep(sleep_time_on_failure)
        return False
 
    def check_url(self, url):
        print 'Checking...' + url + '...',
        try:
            urllib2.urlopen(url, timeout=url_timeout)
        except:
            print 'Failed!'
            return False
        print 'Done it\'s up!'
        return True
 
    def alarm(self):
        print 'Too many failures...'
        while True:
            print 'Alarm...'
            winsound.PlaySound("SystemExit", winsound.SND_ALIAS)
 
    def sleep(self, seconds):
        print 'Sleeping for ' + str(seconds) + ' second(s)...'
        time.sleep(seconds)
        print 'Waking up...'
 
def main():
    SiteMonitor().start()
 
if __name__ == "__main__":
    main()

Save it as sitemonitor.py and run it like:
python sitemonitor.py

Or just double click on the file.

Please note: in order to run this, you will have to install Python:
http://www.python.org/download/

A little Program For Monitoring Your Websites, With Alarm
Comments (1)   Filed under: Python, Server   Posted by: Codehead

Web App Security: XSS Attacks

Today, I saw a funny comment on a website:

<script> alert('0wn3d by X - X') </script>
 
<meta HTTP-EQUIV=Refresh CONTENT="0; URL=Some URL">

In case you don’t know about these types of attacks, an attacker will write this comment on a blog (or any sort of web application) and if the application doesn’t escape it before displaying it, this code will display an alert box and then redirects your visitors to whatever the URL is right away.

So again, if I visit this page, I see the alert box and will be redirected to another page on the Internet.

To prevent this, you will have to escape all user generated content before displaying them on your pages, in PHP:

function html_escape($str) {
   return htmlentities($str, ENT_QUOTES, 'utf-8');
}

In Python:

import cgi
 
# ...
 
def escape_html(value)
   return cgi.escape(value, True)

These types of attacks are called Cross-Site Scripting or XSS:
http://en.wikipedia.org/wiki/Cross-site_scripting

Good Luck :)

Web App Security: XSS Attacks
Comments (0)   Filed under: PHP, Python, Security, Web Development   Posted by: Codehead
Older Posts »