{"id":14643,"date":"2023-07-04T11:39:11","date_gmt":"2023-07-04T08:39:11","guid":{"rendered":"https:\/\/sarzimanli.com\/?p=14643"},"modified":"2023-07-04T23:17:35","modified_gmt":"2023-07-04T20:17:35","slug":"dell-vxrail-expired-cert","status":"publish","type":"post","link":"https:\/\/sarzimanli.com\/index.php\/2023\/07\/04\/dell-vxrail-expired-cert\/","title":{"rendered":"Dell VxRail: Unable to log in to vCenter due to expired certificates"},"content":{"rendered":"\n<p>Log in to vCenter GUI is not possible.&nbsp;<\/p>\n\n\n\n<p>If Web GUI is still available, any log in attempt with correct credentials fails.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_15.jpeg\" alt=\"kA23a0000000BfwCAE_3_0\"\/><\/figure>\n\n\n\n<p>Restart of VCSA services fails.<\/p>\n\n\n\n<p>Restart of services does not bring up all services.<\/p>\n\n\n\n<p><strong>Errors observed:&nbsp; &nbsp; &nbsp;<\/strong><\/p>\n\n\n\n<p>\/var\/log\/vmware\/vpxd-svcs\/vpxd-svcs.log:<\/p>\n\n\n\n<p>2020-06-03T09:31:04.523Z [pool-8-thread-1&nbsp; INFO&nbsp; com.vmware.identity.token.impl.X509TrustChainKeySelector&nbsp; opId=905f6864-c067-4db6-828c-1d59c4b43bf8] Failed to find trusted path to signing certificate &lt;CN=ssoserverSign&gt;<br>sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Cause<\/h4>\n\n\n\n<p>vCenter certificates are expired.<\/p>\n\n\n\n<p>VxRail which was initially built prior to 4.7, may have certificates issued with a lifespan of two years from the date of installation. At the time of writing this article, a VxRail build on 4.7.410 has all certificates with a 10 year lifespan.<\/p>\n\n\n\n<p>Minor version upgrades will not touch the certificates!<\/p>\n\n\n\n<p>For a VxRail which was initially built on 4.5.210&nbsp;and later&nbsp;versions, the certificates have a two-year validity&nbsp;period. Check&nbsp;<a href=\"https:\/\/kb.vmware.com\/s\/article\/79248\" target=\"_blank\" rel=\"noreferrer noopener\">VMware KB article 79248<\/a>&nbsp;to confirm the detailed description.<\/p>\n\n\n\n<p>Use the view certificate in the browser of&nbsp;the log in page of the VCSA to confirm the certificate has expired or list the certificates in the CLI of the PSC\/VCSA with the command from&nbsp;<a href=\"https:\/\/kb.vmware.com\/s\/article\/76719\" target=\"_blank\" rel=\"noreferrer noopener\">VMware KB article 76719<\/a>:&nbsp; &nbsp;&nbsp;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">for i in $(\/usr\/lib\/vmware-vmafd\/bin\/vecs-cli store list); do echo STORE $i; \/usr\/lib\/vmware-vmafd\/bin\/vecs-cli entry list --store $i --text | egrep \"Alias|Not After\"; done<\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"resolution\">Resolution<\/h4>\n\n\n\n<p>This procedure will generate new self-signed certificates on PSC and VCSA.<\/p>\n\n\n\n<p><strong>IMPORTANT<\/strong>:&nbsp;This procedure is intended for single PSC\/VCSA VMs which are maintained through VxRail LCM. For HA \/ ELM \/ Customer deployed VCs &#8211; please open a VMware ticket!<\/p>\n\n\n\n<p><strong>IMPORTANT<\/strong>: Take&nbsp;<strong>OFFLINE&nbsp;<\/strong>snapshots of VRM, PSC, and VCSA!<\/p>\n\n\n\n<p><strong>IMPORTANT<\/strong>: Check if the snapshot creating process has finished without errors!&nbsp;<strong>Do NOT continue without valid snapshots!<\/strong><\/p>\n\n\n\n<p><strong>IMPORTANT<\/strong>:&nbsp;If issues encountered, do not retry without reverting to snapshots!<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Fix PSC:\n<ul class=\"wp-block-list\">\n<li>Reset all Certificates &#8211; Option 8 &#8211; (This will fail but it is&nbsp;expected to)<\/li>\n\n\n\n<li>Start certificate manager:&nbsp; &nbsp; &nbsp;<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-preformatted\"><em>\/usr\/lib\/vmware-vmca\/bin\/certificate-manager<\/em>\n\n<\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Select option &#8220;8. Reset all Certificates&#8221;\n<ul class=\"wp-block-list\">\n<li>Confirm&nbsp; &#8220;Do you wish to generate all certificates using configuration file : Option[Y\/N] ?&#8221;<\/li>\n\n\n\n<li>Enter credentials<br><br>&nbsp;<img decoding=\"async\" alt=\"kA23a0000000BfwCAE_2_0\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_1.jpeg\"><br>&nbsp;<\/li>\n\n\n\n<li>Enter values\n<ul class=\"wp-block-list\">\n<li>leave &#8220;<em>IPAddress<\/em>&#8221; field empty<\/li>\n\n\n\n<li>Enter<strong>&nbsp;FQDN of PSC<\/strong>&nbsp;into &#8220;<em>Hostname<\/em>&#8220;<\/li>\n\n\n\n<li>VMCA &#8220;<em>Name<\/em>&#8221; field is name of new Root CA being created (e.g. &#8220;<em>VxRail CA<\/em>&#8220;)<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Confirm &#8220;<em>Continue operation : Option[Y\/N] ?<\/em>&#8220;<\/li>\n\n\n\n<li>Confirm &#8220;<em>Continue operation : Option[Y\/N] ?<\/em>&#8220;<\/li>\n\n\n\n<li>This operation will fail with:<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p><em>Get site nameCompleted [Reset Machine SSL Cert&#8230;]<\/em><\/p>\n\n\n\n<p><em>g3node-site<br>Lookup all services<br>Get service g3node-site:9ebfb9b2-d202-4d8f-9282-b08f3a317b8f<br>Update service g3node-site:9ebfb9b2-d202-4d8f-9282-b08f3a317b8f; spec: \/tmp\/svcspec_a1hipoqq<br><br><strong>Status : 0% Completed [Reset operation failed]<\/strong><br><br>please see \/var\/log\/vmware\/vmcad\/certificate-manager.log for more information.<br>root@g3psc [ ~ ]#&nbsp;<\/em><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_2.jpeg\" alt=\"kA23a0000000BfwCAE_2_1\"\/><\/figure>\n\n\n\n<p>2. Fix the STS issue &#8211; Download and run fixsts.sh from&nbsp;<a href=\"https:\/\/kb.vmware.com\/s\/article\/76719\" target=\"_blank\" rel=\"noreferrer noopener\">VMware KB 76719<\/a><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_3.jpeg\" alt=\"kA23a0000000BfwCAE_2_2\"\/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Stop all services\n<ul class=\"wp-block-list\">\n<li><em>service-control &#8211;all &#8211;stop<\/em><\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Start all services &#8211; will fail &#8211; expected&#8230;\n<ul class=\"wp-block-list\">\n<li><em>service-control &#8211;all &#8211;start<\/em><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_4.jpeg\" alt=\"kA23a0000000BfwCAE_2_3\"\/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Either wait for the process to time-out or you can stop it when it gets to the &#8220;<em>vmware-vmon<\/em>&#8221; service<\/li>\n<\/ul>\n\n\n\n<p><em>\/usr\/lib\/vmware-vmca\/bin\/certificate-manager<\/em><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Select option &#8220;<em>6. Replace Solution user certificates with VMCA certificates<\/em>&#8220;<\/li>\n\n\n\n<li>Confirm &#8220;<em>Do you wish to generate all certificates using configuration file : Option[Y\/N] ?<\/em>&#8220;<\/li>\n\n\n\n<li>Enter credentials<\/li>\n\n\n\n<li>Deny (enter &#8220;N&#8221; ) for &#8220;<em>certool.cfg file exists, Do you wish to reconfigure : Option[Y\/N] ?<\/em>&#8221; &#8211; as all options were configured above<\/li>\n\n\n\n<li>Confirm &#8220;<em>You are going to regenerate Solution User Certificates using VMCA,&nbsp;Continue operation : Option[Y\/N] ?<\/em>&#8220;<\/li>\n\n\n\n<li>Wait until procedure exits, it will:\n<ul class=\"wp-block-list\">\n<li>generate all certificates<\/li>\n\n\n\n<li>stop the services<\/li>\n\n\n\n<li>start the services<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_5.jpeg\" alt=\"kA23a0000000BfwCAE_2_4\"\/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Confirm if all services are running<\/li>\n<\/ul>\n\n\n\n<p><em>service-control &#8211;all &#8211;status<\/em><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_6.jpeg\" alt=\"kA23a0000000BfwCAE_2_5\"\/><\/figure>\n\n\n\n<p>3.&nbsp;Fix certificates on VCSA<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Stop &amp; start all services &#8211;&nbsp;<strong><u>must<\/u>&nbsp;<\/strong>be done AFTER all PSC services are running&nbsp; &#8211;&nbsp;<strong>IMPORTANT!!!<\/strong>\n<ul class=\"wp-block-list\">\n<li><em>service-control &#8211;all &#8211;stop<\/em><\/li>\n\n\n\n<li><em>service-control &#8211;all &#8211;start<\/em><br>&nbsp;<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_7.jpeg\" alt=\"kA23a0000000BfwCAE_2_6\"\/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Either wait for the process to time-out or you can stop it when it gets to the &#8220;<em>vmware-vmon<\/em>&#8221; service<\/li>\n\n\n\n<li>Reset all Certificates &#8211; option 8<\/li>\n\n\n\n<li>Start certificate manager<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-preformatted\"><em>\/usr\/lib\/vmware-vmca\/bin\/certificate-manager<\/em>\n\n<\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Select option &#8220;<em>8. Reset all Certificates<\/em>&#8220;<\/li>\n\n\n\n<li>Confirm &#8220;<em>Do you wish to generate all certificates using configuration file : Option[Y\/N] ?<\/em>&#8220;<\/li>\n\n\n\n<li>Enter credentials<br>&nbsp;<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_8.jpeg\" alt=\"kA23a0000000BfwCAE_2_7\"\/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Enter PSC IP<\/strong><\/li>\n\n\n\n<li>Enter values<\/li>\n\n\n\n<li>leave &#8220;<em>IPAddress<\/em>&#8221; field empty<\/li>\n\n\n\n<li>Enter<strong>&nbsp;FQDN of VCSA<\/strong>&nbsp;into &#8220;<em>Hostname<\/em>&#8220;<\/li>\n\n\n\n<li>VMCA &#8220;<em>Name<\/em>&#8221; field is name of new Root CA being created (e.g. &#8220;<em>VxRail CA<\/em>&#8220;)<\/li>\n\n\n\n<li>Confirm &#8220;<em>Continue operation : Option[Y\/N] ?<\/em>&#8220;<\/li>\n\n\n\n<li>Confirm &#8220;<em>Continue operation : Option[Y\/N] ?<\/em>&#8220;<br>&nbsp;<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_9.jpeg\" alt=\"kA23a0000000BfwCAE_2_8\"\/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Wait until all certificates are generated and the message &#8220;<em>Reset status : 100% Completed [Reset completed successfully<\/em>]&#8221; appears<br><br><img decoding=\"async\" alt=\"kA23a0000000BfwCAE_2_9\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_10.jpeg\"><br><img decoding=\"async\" alt=\"kA23a0000000BfwCAE_2_10\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_11.jpeg\"><\/li>\n\n\n\n<li>Check if all services are running<\/li>\n<\/ul>\n\n\n\n<p><em>service-control &#8211;all &#8211;status<\/em><\/p>\n\n\n\n<p><img decoding=\"async\" alt=\"kA23a0000000BfwCAE_2_11\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_12.jpeg\"><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Access vCenter GUI<\/li>\n\n\n\n<li>Access by DNS will fail in Chrome due to HSTS &#8211; open the VCSA IP or use another supported browser (e.g. FireFox)<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_13.jpeg\" alt=\"kA23a0000000BfwCAE_2_12\"\/><\/figure>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/supportkb.dell.com\/img\/ka06P000000TRfuQAG\/ka06P000000TRfuQAG_en_US_14.jpeg\" alt=\"kA23a0000000BfwCAE_2_13\"\/><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\">Additional Information<\/h4>\n\n\n\n<p><strong>Notes:&nbsp; &nbsp;&nbsp;<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>ALWAYS take snapshots of System VMs (PSC\/VCSA\/VRM) before following this&nbsp;KB.<\/li>\n\n\n\n<li><strong>This procedure is intended for PSC\/VCSA VMs which are maintained through VxRail LCM.<\/strong><\/li>\n\n\n\n<li>If user has&nbsp;certificates from their own infrastructure, they&nbsp;can replace them at this point again.<\/li>\n\n\n\n<li>After fix for VxRail version 4.7.100+, follow KB article 486959:&nbsp;<a href=\"https:\/\/www.dell.com\/support\/kbdoc\/en-us\/000077894\/vxrail-how-to-replace-certificate-in-vcenter-for-vxrail-appliance\" target=\"_blank\" rel=\"noreferrer noopener\">VxRail: How to manually import vCenter SSL certificate on VxRail Manager<\/a>&nbsp; to import new root certificate into VRM (plugin will not work).<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Log in to vCenter GUI is not possible.&nbsp; If Web GUI is still available, [&#8230;]<\/p>\n","protected":false},"author":1,"featured_media":14583,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-14643","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-vxrail-hci"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/posts\/14643","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/comments?post=14643"}],"version-history":[{"count":2,"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/posts\/14643\/revisions"}],"predecessor-version":[{"id":14653,"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/posts\/14643\/revisions\/14653"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/media\/14583"}],"wp:attachment":[{"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/media?parent=14643"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/categories?post=14643"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sarzimanli.com\/index.php\/wp-json\/wp\/v2\/tags?post=14643"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}