Azure Container Apps: Internal Server Error on provisioning ACA Environment in existing VNet

Product:

Azure Container Apps, Azure Virtual Networks

Problem:

As I was trying to set up ACA in an existing lab VNet, I discovered that some of my operations for creating the ACA Environment were failing with "Internal Server" error. 

Fortunately I've been through this type of obtuse error messaging from Azure before, so I was able to quickly walk back to what seemed to be causing my issue. 

Powershell received the following error:

New-AzContainerAppManagedEnv @EnvArgs
New-AzContainerAppManagedEnv : Internal server error occurred.  correlation ID: <snip>
At line:1 char:1
+ New-AzContainerAppManagedEnv @EnvArgs
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: ({ SubscriptionI...    }
  }
} }:<>f__AnonymousType16`4) [New-AzContainer..._CreateExpanded], Exception
    + FullyQualifiedErrorId : InternalServerError,Microsoft.Azure.PowerShell.Cmdlets.App.Cmdlets.NewAzContainerAppManagedEnv_CreateExp
   anded

ARM deployments received multiple of the following error and then timed out: 

    "properties": {

        "statusCode": "InternalServerError",

        "serviceRequestId": null,

        "statusMessage": "{\"error\":{\"code\":\"InternalServerError\",\"message\":\"Internal server error occurred. correlation ID: <snip>\"}}",

        "eventCategory": "Administrative",

        "entity": "/subscriptions/<snip>/resourcegroups/<rgname>/providers/Microsoft.App/managedEnvironments/<envname>",

        "message": "Microsoft.App/managedEnvironments/write",

    },

 

 

Solution: 

Remove IPv6 address spaces from the VNet. 

 

HTH.  

Terraform quick view of resource schema

 I was getting tired of having to go to Hashicorp's website to view provider arguments lists, because SURELY the arguments list is in the code somewhere handy. As it turns out, it is.

I made a quick bash function to export the schema for a given resource provider

tfschema () {
    if [ $# -ne 2 ]
    then
        echo -e "This function requires exactly two arguments, in order: providername resourcename "
        echo "Try: tfschema  registry.terraform.io/hashicorp/vsphere vsphere_virtual_machine"
        echo "It is required to have already run terraform init with HCL for the provider used in the first argument."
        echo "Try again. You have 1 life remaining. Exiting..."
        return
    fi
    if ! which terraform >/dev/null 2>&1
    then
        echo "terraform not found. Exiting..."
        return
    fi
    
    terraform providers schema --json|jq '.provider_schemas["'"$1"'"].resource_schemas["'"$2"'"].block| .attributes, .block_types|values'
}

Quick Screenshot:

Terraform Azure Devops hosted pipelines

 In setting up my homelab using AZDO, I wanted to run some builds and steps using the Azure Hosted Agent. 

The documentation is a little unclear on this so I imported resources as follows. 

If codeblock output is clipped by css, don't worry, the code is there. Not working on that tonight as I have homelab to set up. 

For given TF code:

resource "azuredevops_agent_pool" "pipelinesagent" {

  name           = "Azure Pipelines"

  auto_provision = true

}

resource "azuredevops_agent_queue" "pipelinesqueue" {

  project_id    = azuredevops_project.project.id

  agent_pool_id = azuredevops_agent_pool.pipelinesagent.id

}

I found the azuredevops_agent_pool ID and the azuredevops_agent_queue ID as follows:

az pipelines queue list

// There were a lot of queues, but I am interested in the last in the output: 

// guids replaced with <guidhere>, get your own.

// 

[

{

    "id":  45,

    "name":  "Azure Pipelines",

    "pool":  {

                 "id":  9,

                 "isHosted":  true,

                 "isLegacy":  false,

                 "name":  "Azure Pipelines",

                 "options":  "none",

                 "poolType":  "automation",

                 "scope":  "<guidhere>",

                 "size":  1

             },

    "projectId":  "<guidhere>"

}

]

Now I can run my imports:

terraform import azuredevops_agent_pool.pipelinesagent 9

terraform import azuredevops_agent_queue.pipelinesqueue <guidhere>/45

And a `terraform apply` shows no changes needed. 

ESXi 7 PCI Passthrough Gigabyte B550 board

 

System:

Gigabyte B550 Aorus Elite Pro

Ryzen 3100

Sandisk 32GB USB sticks

Problem:

After trying to pass through an LSI 9207-8i, subsequent reboots still showed the device in " Enabled / Needs reboot" in ESXi 7.0 and the device was not assignable.

I worked through a few steps:

Disabled the LSI driver so it wouldn't grab the card

esxcli system module set --enabled=false --module=lsi_msgpt2

(can't remember if it was gpt2 or gpt3 though)

Changed system UEFI settings 

Found these on reddit, not sure if they really helped: 

Tweaker -> Advanced CPU Settings -> SVM Mode -> Enabled
Settings -> Miscellaneous -> IOMMU -> Enabled
Settings -> NBIO Common Options -> IOMMU -> Enabled
Settings -> NBIO Common Options -> ACS Enable -> Enable
Settings -> NBIO Common Options -> Enable AER Cap -> Enable

Disable ACS Checks in ESXi

Manage > System > Advanced settings
Set VMkernel.Boot.disableACSCheck to True.

 

 

 Reboot. Done.

ESXI 7 installer error 28 vmx.v00 and resolution

ESXI 7 installer error on Ryzen 3100 machine

System:

Gigabyte B550 Aorus Elite Pro

Ryzen 3100

Sandisk 32GB USB sticks

 Seeing the following error during the boot of the install USB stick, imaged by Rufus from ISO.

CRC error during decompression, Received CRC (<snip>) != calculated CRC (<snip>)

gzip_extract failed for /vmx.v00 (size 0): CRC error

Error 28 (CRC) error while loading module: /vmx.v00

Compressed MD5: <snip>

Decompressed MD5: 0000000000000<snip>

Fatal error: 28 (CRC Error)

Tried:

• Different USB ports
• Different ESXI versions (6.7 produced similar errors) 
• Different USB sticks
• Different partition types (MBR vs GPT)
• Booting from an existing ESXI 6.7 install

Resolution found after these two steps: 

• Disabled CSM (Compatibility Support Mode) in the UEFI settings
• Ensured the machine booted from the UEFI bootloader on the USB stick. For some reason the BIOS wanted to boot from "VMWare USB FDMAP" or something, but there were two additional boot entries that were prefixed with "UEFI: " and that worked like a charm.

 

Searching for the CRC 28 error found zero helpful hits, so hopefully this will help provide relevant resolutions for someone.

Amazon Alexa App / Echo Dot "Failed to authorize link code. Please try again."

This is just consumer gear, but Googling didn't turn up anything at all for the error, so here you go.

Symptoms:
Try to set up a freshly reset Echo device. When joining WiFi, you receive an error in a red banner at the top of the screen that says "Failed to authorize link code. Please try again."

Resolution:
Try from another device running the Alexa app. I tried from my wife's phone (where the Echo Dot order was originally placed) and it worked straight away.

ESXi 6.7u2 update to 6.7u3 failed

Error on my host: Update manager fails to patch the host. When running the manual patch, the host profile update fails.

Error in esxupdate.log:

2019-08-25T00:45:24Z esxupdate: 2099348: root: ERROR: esximage.Errors.InstallationError: ('VMware_locker_tools-light_10.3.10.12406962-14141615', '[Errno 28] No space left on device')

Resolution:
cd /tmp
wget http://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/tools-light/VMware_locker_tools-light_6.5.0-0.23.5969300.vib
esxcli software vib install -f -v /tmp/VMware_locker_tools-light_10.3.10.12406962-14141615.vib

Resume manual patching:
esxcli software profile update -p ESXi-6.7.0-20190802001-standard \ -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml 

esxcli network firewall ruleset set -e false -r httpClient

reboot

Done.