Translate

lunes, 20 de septiembre de 2021

Azure: enlaces útiles de Azure Datafactory




Información General:

Azure Data Factory documentation

https://docs.microsoft.com/en-us/azure/data-factory/



Azure Data Factory Managed Virtual Network

https://azure.microsoft.com/en-us/blog/azure-data-factory-managed-virtual-network/?cdn=disable




Linked services in Azure Data Factory and Azure Synapse Analytics

https://docs.microsoft.com/en-us/azure/data-factory/concepts-linked-services


Private Endpoints: 


Azure Private Link for Azure Data Factory

https://docs.microsoft.com/en-us/azure/data-factory/data-factory-private-link



Managed Virtual Networks and Private Endpoints in Azure Synapse and Azure Data Factory 

https://mitchellpearson.com/2021/01/11/managed-virtual-networks-and-private-endpoints-in-azure-synapse-and-azure-data-factory/

 


Azure Data Factory Managed Virtual Network (preview)

https://docs.microsoft.com/en-us/azure/data-factory/managed-virtual-network-private-endpoint



Copy data securely from Azure Blob storage to a SQL database by using private endpoints

https://docs.microsoft.com/en-us/azure/data-factory/tutorial-copy-data-portal-private



Tutorial: How to access SQL Managed Instance from Data Factory Managed VNET using Private Endpoint

https://docs.microsoft.com/en-us/azure/data-factory/tutorial-managed-virtual-network-sql-managed-instance


Tutorial: How to access on-premises SQL Server from Data Factory Managed VNet using Private Endpoint

https://docs.microsoft.com/en-us/azure/data-factory/tutorial-managed-virtual-network-on-premise-sql-server



Referencias:


Azure Info Hub: Azure Data Factory

https://azureinfohub.azurewebsites.net/Service/Documentation?serviceTitle=Azure%20Data%20Factory

jueves, 20 de mayo de 2021

Git: Mostrar los archivos modificados en tu último commit

    


Hoy traemos la típica funcionalidad que siempre se nos olvida y la traigo aqui como recordatorio, el caso de uso es el típico que estoy trabajando en una rama y tengo que cambiar a otra para ponerme con otra cosa urgente.

Podríamos usar el stash, pero muy sucio tendría que estar el código para usarlo, yo prefiero hacer un commit y dejar la rama limpia y cambiarme a trabajar con la rama urgente.

git commit -m "cambios parciales"

git checkout miRamaUrgente

Después cuando vuelves a la rama original después de varios días o simplmente porque quieres ver los archivos modificados porque no te acordabas o simplemente porque quieres ver lo que ha modificado algún otro compañero.

 

git checkout miRamaOriginal

git log --name-only

Si quiero sólo pero último commit

git log -1 --name-only

martes, 11 de mayo de 2021

Azure: Start / Stop AKS Cluster con API, ansible y CLI


No hace mucho Microsoft ha permitido la opción de parar los AKS, esta era una funcionalidad que se echaba mucho de menos porque una vez levantado un AKS ya no se podía para y seguía costando dinero mes a mes y sólo teníamos 2 opciones para no tener tanto coste a final de mes.


  • Destruir del aks
  • Modificar el número de nodos y/o la capacidad de las máquinas.

Pero con esta nueva opción podemos parar el AKS entero y mantener la configuración que tenemos del mismo sin que nos cueste dinero. 

Esta opción no está disponible a través del portal las opciones que tenemos son las siguientes:

  • 1 .  De manera manual trabajando con AZ CLI, para ello tienes que tener instalado la versión de AZ CLI de tu sistema operativo, ya sea windows o linux. Yo trabajo con linux y actualmente estoy usando la version azure-cli 2.21.0
    • az login: introducir las credenciales en el navegador
    • az account list: para optener el listado las subscriones disponibles
    • az account set -s [subcription_id o subscription_name] para situarnos en la subscripción del aks
    • Parar AKS: az aks stop --name myAKSCluster --resource-group myResourceGroup --subscription mySubscription
    • Arrancar AKS: az aks start --name myAKSCluster --resource-group myResourceGroup --subscription mySubscription
    • Comprobar el estado de AKS:  az aks show --name myAKSCluster --resource-group myResourceGroup --subscription mySubscription
    • El campo a revisar es powerState
--
  "powerState": {
    "code": "Stopped"
  },
--

                Nota: Esta opción puede ejecutarla un usario nominal  

  • 2. A mi que me gusta más tener procesos automatizados prefiero usar ansible para automatizar esta tipo de llamadas a la API en vez de usar los comandos desde la consola. Para ello necesitamos un appregistration con permisos RBAC sobre el resource group (RG) o sobre el propio AKS y lo haremos dos sencilos pasos:
    • Generar el token de acceso en base a las credenciales en base al appregistration (service principal) que tiene acceso al recurso de la subscripción.
- name: Get token in order to obtain  aks credentials for admin cluster user
  uri:
    url: "https://login.microsoftonline.com/{{ TENANT }}/oauth2/token"
    method: POST
    body_format: form-urlencoded
    return_content: true
    headers:
      Content-Type: application/x-www-form-urlencoded
    body:
      grant_type: client_credentials
      client_id: "{{ CLIENT_ID }}"
      client_secret: "{{ CLIENT_SECRET }}"
      resource: 'https://management.azure.com/'
  register: _response
  failed_when: _response.json is not defined or 'access_token' not in _response.json

- name: set token
  set_fact:
    _api_manager_token: "{{ _response.json.access_token }}"
  failed_when: _api_manager_token is not defined

--
    • Llamada a la API de AKS para parar/arracar el cluster con las opciones de power_action = stop o power_action = start

- name: "Power {{ power_action }} the aks {{ aks_name }}"
  uri:
    url: "https://management.azure.com/subscriptions/{{ aks_subscription_id }}
          /resourceGroups/{{ aks_resource_group }}/providers/Microsoft.ContainerService
          /managedClusters/{{ aks_name }}/{{ power_action }}?api-version=2020-09-01"
    body_format: json
    method: POST
    return_content: true
    status_code: [200, 202]
    headers:
      Content-Type: application/json
      Authorization: "Bearer {{ _api_manager_token }}"
    body:
  register: _api_response

- name: "Wait until the {{ aks_name }} is {{ power_action }}"
  uri:
    url: "{{ _api_response.azure_asyncoperation }}"
    method: GET
    status_code: [200]
    return_content: true
    headers:
      Content-Type: application/json
      SubscriptionId: "{{ aks_subscription_id }}"
      Authorization: "Bearer {{ _api_manager_token }}"
  until: _info.json.status == "Succeeded"
  retries: 50
  delay: 30
  register: _info
  failed_when: ( _info.status != 200 or _info.json is not defined or 'status' not in _info.json )


--

Referencias:

AKS stop API

AKS start API

AZ CLI start/stop

martes, 4 de julio de 2017

MySql tinyInt1isBit = false

Hoy una entrada rápida que puede salvar más de un problema una vez que tenemos nuestra base de datos ya definida o simplemente la hemos heredado de otra persona o proveedor.

Que ocurre si tenemos definido un campo en una tabla MySQL como tinyint, este es un campo que se usa para almacenar datos de 8bits es decir valores de 0 a 255.

Pero normalmente se utiliza para valor es boolean tipo true false. En mi caso hoy hemos tenido un problema porque tenemos una columna definida con este tipo de datos y en la tabla tenemos valores 0, 1, 2, etc.... El problema ha surgido porque el driver de mysql php ha traducido el valor correctamente pero el de java no lo ha hecho.

El driver de php traduce correctamente el valor 2, pero en Java nos devuelve el valor true. 

Curioseando por la web hemos visto que si en los parámetros de conexión de MySQL añadimos la siguiente cadena de conexión en la conexión java tinyInt1isBit=false el valor será interpretado correctamente como un 2.

Esto a más de uno le puede salvar de más de un problema.


miércoles, 18 de enero de 2017

Jboss datasources: how to set autocommit to false by default with example

Como cambiar el valor de autocommit  en los data sources de JBOSS  EAP 6.4 con ejemplo práctico:

In this post i´m going to resolve a problem with the jboss datasources, when you open a connection and get it throught a jboss datasource the autocommit mode is setted to true by default.

When you develop a new application where you can control all your code it is not a problem, but when you have  inherited code that you cant change it, for example in a migration from a weblogic to a jboss server it could be a real problem


In my case I had java code inherited from weblogic and in some place of the code i was suffering problems with the autocommit state of my jdbc connection. I could not do commit or rollback becasuse the autocommit was setted to true, and it throw the next exception

java.slq.SQLException: you cannot rollback with autocommit set!

This is because in weblogic 10.3 the jdbc connections have a extrange behavior because if you try to do a rollback with autocommit = true it has no effect butit doesn´t thow an exception. When you migrate this code to Jboss is a real problem that you have to handle.

In order to find a solution I have debugged the inherited code to looking for the classes that has been used to get the datasource connection. I found that the datasource class is WrapperDataSource 



In my case the solution was to create a Wrapper of the class WrapperDataSource in order to add the code to setAutoCommit(false) in the  getConnetion methods, in order to get the connection with autocommit setted to false by default, when the application instace a jdbc connection from Jboss server.


To do this we need to create a java project with the sources and the compiled classes of the this jar: ironjacamar-jdbc-1.0.31.Final-redhat-1.jar in order to modify and compile the WrapperDataSource class. This is the class that Jboss uses to create a new JDBC connection.

You can find this sources in the nex  link or in my github repository in the lib folder.

https://maven.repository.redhat.com/techpreview/all/org/jboss/ironjacamar/ironjacamar-jdbc/1.0.31.Final-redhat-1/

Also we need some jars in order to compile the WrapperDataSource class, the jars are those:

  • ironjacamar-core-api-1.0.31.Final-redhat-1.jar
  • ironjacamar-jdbc-1.0.31.Final-redhat-1.jar
  • jboss-connector-api_1.6_spec-1.0.1.Final-redhat-3.jar
  • jboss-logging-3.1.4.GA-redhat-2.jar
  • jboss-transaction-api_1.1_spec-1.0.1.Final-redhat-3.jar


All jars are in the Jbos EAP 6.4 distribution or in my github repository code too.

https://developers.redhat.com/products/eap/download/



Now in the java project with all the jars (added to the poject build path) and the source files we can make the modification that we need. We only need to create the package org.jboss.jca.adapters.jdbc and copy the class WrapperDataSource.java from the source files into it. If you need to modify more classes take it from the source code, copy in the project and modify it.



Next step is to look up the getConnection Methods and add the setAutocommit(false), after the connection creation,


Be carefull because in this version exists two getConnection methods and i have to modify both.

In next step you have to compile this class and mix with the original classes and create the new jar with my changes. In order to do this i have create an ant script (buidl.xml) into the ant folder which unzip the orginal, jar, compile the modified class and add to the new jar.

--

<?xml version="1.0"?>

<project name="WrapperDataSource" default="build" basedir=".">

 <property file="build.properties"/>
 
 
 <target name="clean" description="Deletes compiled and generated code">
         <delete dir="${build.dir}" />
 </target>
 
 <target name="build" depends="clean" description="build Wrapper">
  <mkdir dir="${build.dir}" />
  <unzip src="${build.dir}/../lib/ironjacamar-jdbc-1.0.31.Final-redhat-1.jar" dest="${build.dir}"/>
  <path id="lib.path.ref">
      <fileset dir="${lib.path.ref}" includes="*.jar"/>
  </path> 
  <javac srcdir="${source.dir}" destdir="${build.dir}" classpathref="lib.path.ref" target="1.6" 
 source="1.6" debug="true"/> <jar   destfile="${jar.dir}/${jar.name}"
         basedir="${build.dir}"         
  />
 </target></project>
--

To build the JAR with the new modified class, be sure to have Apache Ant installed, go to ant directory and execute the "ant" command.

The result shoud be:


The result is in the target/dist foder where you can find the new MyJbossWrapperDataSource.jar



Finally the last step consists in change the original ironjacamar-jdbc-1.0.31.Final-redhat-1.jar by the new MyJbossWrapperDataSource.jar modifiying the Jboss libray module.

To do this action you have to go to your JBOSS EAP6 installation and locate the next path:

[JBOSS_HOME]\modules\system\layers\base\org\jboss\ironjacamar\jdbcadapters\main\

Copy the new MyJbossWrapperDataSource.jar to this location



and modify the module.xml commenting the old jar file and adding the new MyJbossWrapperDataSource.jar

--
<module xmlns="urn:jboss:module:1.1" name="org.jboss.ironjacamar.jdbcadapters">
    <properties>
        <property name="jboss.api" value="private"/>
    </properties>

    <resources>
        <!--resource-root path="ironjacamar-jdbc-1.0.31.Final-redhat-1.jar"/>-->
        <resource-root path="MyJbossWrapperDataSource.jar"/>
        <!-- Insert resources here -->
    </resources>

    <dependencies>
        <module name="javax.api"/>
        <module name="javax.resource.api"/>
        <module name="javax.validation.api"/>
        <module name="org.hibernate.validator"/>
        <module name="org.jboss.as.naming"/>
        <module name="org.jboss.as.transactions"/>
        <module name="org.jboss.common-core"/>
        <module name="org.jboss.jboss-transaction-spi"/>
        <module name="org.jboss.ironjacamar.api"/>
        <module name="org.jboss.logging"/>
        <module name="org.jboss.threads"/>
        <module name="javax.xml.stream.api"/>
    </dependencies>
</module>

--

You can download my eclipse project from github in the next link, if you want to get the code with this change appied, or make more changes in your jboss datasource.

https://github.com/antuansoft/MyJbossWrapperDataSource

Happy Wrapping!!!


martes, 27 de diciembre de 2016

Weblogic 10.3.6: local queue creation

Weblogic 10.3.6: Creación de colas locales


Sometimes in weblogic server is needed to create some local queues in order to process different types of messages.

This is the steps needed to create a very basic queue in weblogic.

The main objetive is to create one queue in weblogic.

1 - JMS Server creation: 

This is a basic container for de the queues.

  • Go to services -_> Messaging-->Jms Servers
  • Press New and create a new JMS Server: It is the container for queues.
  • Name the new JMS, por example: MyJmsServer and press next.

  • Finally select the server where you want to deploy your Jms Server



  • The final result is that you have created a new JmsServer associated to your Server.




2 - JMS Module creation: 

This step is to create a JMS module where you can assign resources and configure your queues. This module is similar to a J2EE module where configure your jndi resources.


  • Go to JMS Modules and create a new module, for example CSBModule, this is a module related to my CSMB project.

  • Name the module.

  • Asign to a server.
  • The final result is a JMS module asociated to the Server.



3 - Subdeployment creation: 

A subdeployment is a mechanism by which JMS module resources (such as queues, topics, and connection factories) are grouped and targeted to a server resource (such as JMS servers, server instances, or cluster).

Is a place where all the queues and resources which we are going to create will be placed, we are going to asociate to the JMS Server created in step 1.


  • Go to the new JmsModule creante and go to subdeployments tab and create new one


  • Name the subdeployment, for example AdminServerJMS


  • Associate it to the JMS Server create in step 1.

  • The result is a subdeployment associated to the JMS Server and through this server to the admin server.




4 - Conection Factory creation: 

In order to create a queue, the first step is to create a connection factory and after that the queue and asociate it to this factory.

A connection factory is a resource that enables JMS clients to create connections to JMS destinations.

  • Go to JMSModules --> CSMBModule and press new 

  • Create the connection factory 

  • Name it  (CFCSMB) and give a jndi name (jms/CFCSMB)

  • Asociate to the server.

  • The final result is a connection factory like that:


5 - Queue creation: 

Now we are ready to create the queue

  • Go to JMSModules --> CSMBModule and press new 
  • Create the new queue

  • Name it (CSMB_BT_IN) and give a jndi name (jms/CSMB_BT_IN)

  • Associate it to the sub-deployment and target to the JMS Server.

  • The final result is like that


This are all the steps needed to create a Queue in weblogic 10.3.6 associated to a JNDI-NAME.




jueves, 15 de diciembre de 2016

Weblogic Jboss Migration issue 1: javax.naming.NameNotFoundException: env/jmx/runtime



Migración Weblogic - Jboss problema 1: obtener el Servidor MBeans


I my new job, we have to migrate original web applications deployed in a Weblogic 10.6.3 Application Server to Jboss EAP6 (Jboss AS7).

This post is about the first problem that i have found. The problem is how to instantate the MBeanServer object that containts all my beans.


The slice of code that instantate a the MBeanServer in weblogic is like that:

//Weblogic way
InitialContext ctx = new InitialContext();
this.mbs = ((MBeanServer)ctx.lookup("java:comp/env/jmx/runtime"));

If you deploy this code in a JBoss, the common error that appears in log is the next one:
javax.naming.NameNotFoundException: env/jmx/runtime

The way to solve this issue is change the context lookup for a FactoryManager 
to get the right MBeanServer.

//JBoss way
import java.lang.management.ManagementFactory;
//another code
//another code
//.....
this.mbs=ManagementFactory.getPlatformMBeanServer();

The good news is that this way works both in Jboss and Weblogic.