sql server - best practice to load multiple client data into Hadoop -


we creating poc on hadoop framework cloudera cdh. want load data of multiple client hive tables.

as of now, have separate database each client on sql server. infrastructure remain same oltp. hadoop used olap. have primary dimension tables same each client. client database has exact same schema. these tables have same primary key value. till now, fine have separate database client. trying load multiple client data same data container (hive tables). have multiple row same primary key value if load data directly hive multiple sql server databases through sqoop job. thinking use surrogate key in hive tables hive not support auto increment can achieved udf.

we don't want modify sql server data it's running production data.

a. standard/generic way/solution load multiple client data hadoop ecosystem?

b. how primary key of sql server database table can mapped hadoop hive table ?

c. how can ensure 1 client never able see data of other client?

thanks

@praveen: use mappers overcome downtime each clients data hadoop servers, client data holds primary keys in case. use best use of partitions each client , respect date partition. have implement tde zone hdfs file location, before start sqoop import. *tde: trasparent data encryption zone, best practice secured zone client data.


Comments

Post a Comment

Popular posts from this blog

php - Wordpress website dashboard page or post editor content is not showing but front end data is showing properly -

Image
recently have uploaded wordpress site onto server. have uploaded wordpress site in sub-folder under root folder ../root(domain root folder)/wordpress (where host wordpress folder). now problem content data in page or post in dashboard of wordpress admin panel not showing - see image. but data showing in front view - see image. if not browser cache issue , not issue happened tinymce editor during upgrade; try add config file: define('concatenate_scripts', false);
Read more

How to get the ip address of VM and use it to configure SSH connection dynamically in Ansible -

i trying ip address of vm , trying use establish ssh vm. following script: --- - hosts: localhost become: yes connection: local gather_facts: false serial: 1 vars_files: - createvmvars.yml tasks: - name: gathering vm info. vsphere_guest: vcenter_hostname: "{{vcenter_hostname}}" username: "{{vcenter_username}}" password: "{{vcenter_password}}" guest: "{{guest_name}}" vmware_guest_facts: yes register: var - debug: msg="{{var.ansible_facts.hw_eth0.ipaddresses[0]}}" - name: establishing ssh connection. script: /home/shasha/devops/scripts/ssh_configure.sh "{{var.ansible_facts.hw_eth0.ipaddresses[0]}}" following content of ssh_configure.sh : #!/bin/sh ssh-keygen -t rsa ssh-copy-id $1 but when running playbook, getting following error while executing third task: error! 'var' undefined but debug module(second task) printin
Read more

javascript - Get parameter of GET request -

i make request: createxhrrequest( "get", fileurl, function( err, response ) { if( err ) { alert( "error get!" ); } alert(response); }); and full response: { "status" : "ok", "message" : "jvberi0xljqkjdduxdgkmy how can jvberi0xlj ? i try response.message , response["message"]. nothing works. just simple access this: var json = json.parse(response); json.message
Read more