Copying data from S3 to AWS redshift using python and psycopg2 -


i'm having issues executing copy command load data s3 amazon's redshift python.
have following copy command: 

copy moves 's3://<my_bucket_name>/moves_data/2013-03-24/18/moves' credentials 'aws_access_key_id=<key_id>;aws_secret_access_key=<key_secret>' removequotes delimiter ',';

when execute command using sql workbench/j works expected, when try execute python , psycopg2 command pass ok no data loaded , no error thrown.
tried following 2 options (assume psycopg2 connection ok because is):

cursor.execute(copy_command)   cursor.copy_expert(copy_command, sys.stdout)

both pass no warning yet data isn't loaded

ideas?

thanks

i have used exact setup (psycopg2 + redshift + copy) successfully. did commit afterwards? sql workbench defaults auto-commit while psycopg2 defaults opening transaction, data won't visible until call commit() on connection.

the full workflow is:

conn = psycopg2.connect(...) cur = conn.cursor() cur.execute("copy...") conn.commit()

i don't believe copy_expert() or of cursor.copy_* commands work redshift.


Comments

Post a Comment

Popular posts from this blog

php - Wordpress website dashboard page or post editor content is not showing but front end data is showing properly -

Image
recently have uploaded wordpress site onto server. have uploaded wordpress site in sub-folder under root folder ../root(domain root folder)/wordpress (where host wordpress folder). now problem content data in page or post in dashboard of wordpress admin panel not showing - see image. but data showing in front view - see image. if not browser cache issue , not issue happened tinymce editor during upgrade; try add config file: define('concatenate_scripts', false);
Read more

How to get the ip address of VM and use it to configure SSH connection dynamically in Ansible -

i trying ip address of vm , trying use establish ssh vm. following script: --- - hosts: localhost become: yes connection: local gather_facts: false serial: 1 vars_files: - createvmvars.yml tasks: - name: gathering vm info. vsphere_guest: vcenter_hostname: "{{vcenter_hostname}}" username: "{{vcenter_username}}" password: "{{vcenter_password}}" guest: "{{guest_name}}" vmware_guest_facts: yes register: var - debug: msg="{{var.ansible_facts.hw_eth0.ipaddresses[0]}}" - name: establishing ssh connection. script: /home/shasha/devops/scripts/ssh_configure.sh "{{var.ansible_facts.hw_eth0.ipaddresses[0]}}" following content of ssh_configure.sh : #!/bin/sh ssh-keygen -t rsa ssh-copy-id $1 but when running playbook, getting following error while executing third task: error! 'var' undefined but debug module(second task) printin
Read more

javascript - Get parameter of GET request -

i make request: createxhrrequest( "get", fileurl, function( err, response ) { if( err ) { alert( "error get!" ); } alert(response); }); and full response: { "status" : "ok", "message" : "jvberi0xljqkjdduxdgkmy how can jvberi0xlj ? i try response.message , response["message"]. nothing works. just simple access this: var json = json.parse(response); json.message
Read more