Demystifying .NET Core SDK versions

Last night I was developing some code in .NET Core for the Dutch Azure Meetup and I was (again) encountering versioning problems and unexpected behavior.

I found this blog post which describes in some way the same problem I had.

Update 13-01-2017: I found this blog which also describes very well the issue.

What the blog post says is that if you use:

dotnet new

and you want use a specific version of .NET Core SDK you should add a global.json file (before executing the “dotnet new” command) like this:

{
 "sdk": {
 "version": "1.0.0-preview2-003121"
 }
}

The SDK’s in Windows are installed in the Program Files folder (or Program Files (x86) if you have a 32bit Windows) so let’s take a look there:

programfiles
The following SDK’s are installed om my machine:

    • 1.0.0-preview2-003131
    • 1.0.0-preview2-003133
    • 1.0.0-preview2-003156
    • 1.0.0-preview4-004233
  • 1.0.0-preview2-1-003177

But how I know which SDK belongs to which .NET Core version? Continue reading “Demystifying .NET Core SDK versions”

Solve 401 (Unauthorized) error when restoring packages using VSTS Package Management and .NET Core

For the quick answer jump directly to the conclusion

Yesterday I was setting up the build for an ASP .NET Core (Web API) application I wrote, this application was using a package from the VSTS Package Management repository.

To setup this build I was using the new dotnet Core tooling (in preview) which is available when creating a new Build Definition:

createbuilddef

Continue reading “Solve 401 (Unauthorized) error when restoring packages using VSTS Package Management and .NET Core”

Setting up VSTS with ARM Templates

In this post I want to show you what I think it’s the best way to setup VSTS working with Azure Resource Manager Templates.

Introduction

At the customer I am currently working for, we are setting up a new Azure Big Data ingestion environment and we wanted to do it using the Infrastructure as Code approach. With Azure this obviously goes with ARM Templates.

For source control, build and deployment we use Visual Studio Team Services (VSTS).

About VSTS, Build and Release Management

I have seen different setups with VSTS, some of them where the deployment take place from the build, or directly in Release Management without a build.

My approach is to have a clear separation of concerns between the Build and the Release Management.
The Build is for compiling, (Unit) Testing and creating artifacts for the deployment.
The Release Management’s responsibility is for deploying the artifacts created during the Build process. Continue reading “Setting up VSTS with ARM Templates”

VSTS Task delete Resource Group if empty

When testing deployments of Azure RM resources (ie. using the Test-AzureRmResourceGroupDeployment powershell cmdlet) an empty Resource Group is created.
This Resource Group is often not needed at this point and can be deleted after you are done testing the deployment of the resources.
For deleting the Resource Group you can use the VSTS task delete resource group if empty.
This tasks deletes the Resource Group only if it does not contains any resources.

 

Dutch Azure Meetup

meetup logoThis week Pascal Naber, Sander Molenkamp and I started the Dutch Azure Meetup: http://www.dutchazuremeetup.nl

Azure offers a lot of services and features, and this grows almost daily. It’s hard to be up-to-date on all the Azure features. This meetup gives you the opportunity to learn about Azure in a practical way.

We want to offer a platform for Azure enthusiasts, where you can get hands-on with the technology, talk with and learn from each other. The meetup will not handle customer cases in theory but instead we talk about deep technical details, real world experiences and best practices.

Continue reading “Dutch Azure Meetup”

Get subscription id in an Azure Resource Manager Template

Recently I was writing an Azure Resource Manager Template to deploy a Web App. The Web App needed some Application settings (like connection strings, etc..) which I wanted to provision during the Resource Manager Template deployment. One of the settings needed was the Azure subscription id where the Web App was created.
I didn’t want to hardcode the subscription id, or provide it through a parameter (which is a way to postpone the hard-coding), so I did a little research and I found the solution.

In Azure RM Template you can use Resource Functions which can evaluate during the deployment run-time settings.

The one I needed was subscription()  which has the following structure:

{
    "id": "/subscriptions/#####",
    "subscriptionId": "#####",
    "tenantId": "#####"
}

This means you can use the function like this:

"parameters": { 
  "siteName": {
      "type": "string"
  }
},
"resources": [
   {
      "apiVersion": "2014-06-01",
      "name": "[parameters('siteName')]",
      "type": "Microsoft.Web/Sites",
        "properties": {
            "subscriptionid": "[subscription().subscriptionId]",
            "AzureWebJobsDashboard": "xxxx",
            "AzureWebJobsStorage": "xxxx"
        }
   }
]

There are more nice functions you may need to use like: resourceGroup()resourceId ([resourceGroupName], resourceType, resourceName1, [resourceName2]…).

HDInsight error with external metastore (Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient)

Recently I was working to setup a Big Data environment in Azure.
From Azure Data Factory I was spinning up an on-demand HDInsight Cluster with an external metastore.

Unfortunately I was always getting the following error: “Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient”.
After contacting Microsoft for support about this error they found the error was caused by a known Hadoop bug:
https://issues.apache.org/jira/browse/HIVE-12536

In short the error was caused by having dashes (-) in the name of the metastore database. After removing the dashes the problem disappeared and I was able to create the on-demand HDInsight cluster.

An excerpt of the error log, the name of my metastore was db-metastore-p:


Logging initialized using configuration in file:/C:/apps/dist/hive-0.14.0.2.2.9.1-1/conf/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/apps/dist/hadoop-2.6.0.2.2.9.1-1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/apps/dist/hbase-0.98.4.2.2.9.1-1-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:445)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:619)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1483)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:63)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:73)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2743)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2762)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:426)
... 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1481)
... 13 more
Caused by: javax.jdo.JDOUserException: Could not create "increment"/"table" value-generation container db-metastore-p.dbo.SEQUENCE_TABLE since autoCreate flags do not allow it.
NestedThrowables:
org.datanucleus.exceptions.NucleusUserException: Could not create "increment"/"table" value-generation container db-metastore-p.dbo.SEQUENCE_TABLE since autoCreate flags do not allow it.

Access Synology DSM from the internet with a Reverse Proxy

Update:

For DSM 5 the paths are changed:

  • DSM 4: /usr/syno/etc/httpd-ssl-vhost.conf-user
  • DSM 5: /etc/httpd/sites-enabled-user/httpd-ssl-vhost.conf-user

I want to be able to access my Synology Disk Station Management (DSM) from the internet in a secure way through https.

DSM is standard running on the Synology in https mode on port 5001, so we need to setup a reverse proxy in apache:

Access the Synology with SSH using Putty, en type the following commands in the shell prompt to navigate to the right folder:

For DSM 4:

cd /usr/syno/etc/

For DSM 5:

cd /etc/httpd/sites-enabled-user/

Continue reading “Access Synology DSM from the internet with a Reverse Proxy”

Windows 8 keyboard shortcuts summary

Few days ago i was reading a nice post from Scott Hanselman’s blog about Windows 8 enhanced keyboard shortcuts

Here is a summary of the keys:

  • Win+E – Explorer
  • Win+R – Run
  • Win+Break – System Info
  • Win+D – Desktop (I use this constantly if I get trapped outside the Desktop world)
  • Win+Plus or Win+Minus (no shift) – Magnifier/Zoom In and Out
  • Alt-Tab – Switch between Apps
  • Win-Tab – Switch between Full Screen Apps
  • Win+W – Search Control Panel and Settings stuff
  • Win+F – Search Files
  • Esc – Yes, escape. No joke. If you hit the Windows Key, you can hit Escape to go back.
  • Win+C – Charms (right side menu) then arrows to move and enter to launch.
  • Win+K – Devices
  • Win+I – Settings in any app plus brightness, network and other useful system features
  • Win+Arrows – Snap desktop apps to the sides
  • Win+(period) – Alternate sides to “snap” Metro Apps. Add shift to reverse it.
  • Win-X – Admin Stuff. It’s REALLY useful.
  • Win+PageUp or Win+PageDn – Move full screen apps to other monitors. Super useful for News apps and Readers. The Start Screen can be moved as well.

Blog at WordPress.com.

Up ↑