‘An expected field was not found or retrieved properly’ in ArcGIS Pro

Recently I found an interesting issue while upgrading some of the ArcGIS Desktop arcpy scripts to ArcGIS Pro. I was expecting a smooth update by simply running the 2to3 tool. However, an error stops me. The script raises ‘An expected field was not found or retrieved properly’ while running Copy Features tool.

Most of the answers from google were about invalid field names in SQL expression, which means there is at least one join on the source layer. However, in my case, although there was a join, it was later dropped before Copy Feature. I then manually repeated the procedure in ArcGIS Pro and found the same error happened after Remove Join during opening the attributes table of the layer.

I was think what can possibly be wrong. My logic was simple in the script as presented in the following.

1. Creating a table using Frequency tool.
2. Creating a table view using Make Table View tool with where clause.
3. Creating a feature layer using Make Feature layer tool without any where clause applied.
4. Join the feature layer to the table view, join type is KEEP_COMMON.
5. Select all by using Select Layer by Attribute, where clause “1=1”.
6. Remove join.
7. Copy Feature.

The issue happens after remove join obviously. I used arcpy.da.Describe() to inspect the layer and eventually found the issue. The whereClause of the layer is not None. It seems it get the where clause from the Add Join tool but does not remove it while the join is dropped. However, it works fine in ArcMap.

Solution? It is easy enough to get a workaround. Instead of creating a Table View, exporting the table to another table with the same where clause. Then use the new table in the rest of the steps.

The versions of ArcMap and ArcGIS Pro I used in the testing are 10.6.1 and 2.2.4 respectively.

‘General Function failure’ during ‘Share as –> Map Package’ in ArcMap

Recently I encountered the ‘general function failure’ message while running the share as map package tool. A ‘item with the same path name already exists’ error message appears when I switched to the Package Map tool. The map document was not corrupted and all the layers used the same credential to read from a Sql server SDE database.

After spending some time googling, I found some people were struggling over the same issue and suggesting removing/replacing formerly join features is a solution. I double checked my mxd and was not able to find any feature joins. Another guy suggested try the Consolidate Map tool. I tried and it failed with the same ‘item with the same path name already exists’ error message. However, the failed tool didn’t delete the temporary file geodatabase, which gave me more hinds to solve the issue. I found it encountered the error while trying to download the 3rd feature class. It worked when I deleted the last 3 feature class or the first 2 feature class. If I move the last 3 feature class to the top, then the tool failed while trying to download the 4th feature class. It was obvious that the first two feature class belonged to a group while the last three belonged to another. That’s when I thought about the ‘Set Data Sources’ tool. The screen shot shows that the top two feature class were using the same data source while the last three used another. Both sde connection files used exactly the same login credential. So the solution become obvious. You can either make all the feature class use the same data source (.sde connection file) or update one data source with a different credential.

The solution is tested on ArcGIS Desktop 10.5.1 and SDE 10.2.2.

Update 01
When your mxd has feature joins, make sure you use the same data source (.sde connection file) when you intend to use the same database credential.

Update 02
Some commen sense to avoid having the error.

1. Avoid having layers in different schema but stored in the dataset with the name. For example, when having layers in both ‘db1.schema1.dataset1’ and ‘db1.schem2.dataset1’, the tool will try to create two ‘dataset1’ dataset in the output file geodatabase, which causes error.
2. Avoid having layers in different schema but have the same feature class name. For example, the data source for layer 1 is ‘db1.schema1.fc1’ and for layer2 is ‘db1.schema2.fc1’, while running the tool, it will try to create two fc1 in the output file geodatabase, which leads to an error. However, this error is slightly different. It reads ‘the table already exists’.
   

Update 03
When using python to run script that includes the ‘Package Map’ tool, make sure to use the 32 bit python.exe if you have ArcGIS Desktop Background Geoprocessing installed.

Update 04
Further discussion following Update 02.
If you cannot avoid the situation described in Update 02, there is a solution.
Try use different login credentials for each schema which has the feature class with the same name. ‘Package Map’ tool creates one file geodatabase for each data source (.sde connection file) used in the map document.

GeoEvent Server 10.5.1 Failed to start after installation/upgrading

After GeoEvent Server 10.5.1 installtion/upgrading, the geoevent manager page shows a 404 error.

After further inspection in the karaf.log file located in “C:\Program Files\ArcGIS\Server\GeoEvent\data\log”, it is clear that the server stops GeoEvent Server from downloading a couple of JAR files from http://repo1.maven.org/maven2/.  See attached log.

2017-09-12 12:35:28,638 | WARN | FelixStartLevel | AetherBasedResolver | 12 – org.ops4j.pax.logging.pax-logging-api – 1.8.1 | Error resolving artifactorg.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-expression:jar:3.2.11.RELEASE_1:Could not transfer artifact org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-expression:jar:3.2.11.RELEASE_1 from/to central (http://repo1.maven.org/maven2/): Connect to repo1.maven.org:80 [repo1.maven.org/151.101.52.209] failed: Connection refused: connect
shaded.org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-expression:jar:3.2.11.RELEASE_1 from/to central (http://repo1.maven.org/maven2/): Connect to repo1.maven.org:80 [repo1.maven.org/151.101.52.209] failed: Connection refused: connect

So the solution becomes clear. Try to figure out why the server blocks the http request and fix it, restart GeoEvent Server and it should work.

If you don’t want to change server or firewall security settings, there is a manual solution.

1. Download the following JAR files.
   spring-expression and spring-context-support.
2. Stop GeoEvent Server.
3. Copy spring-expression jar to ‘C:\Program Files\ArcGIS\Server\GeoEvent\system\org\apache\servicemix\bundles\org.apache.servicemix.bundles.spring-expression\3.2.11.RELEASE_1’ folder.
4. Copy spring-context-support jar to ‘C:\Program Files\ArcGIS\Server\GeoEvent\system\org\apache\servicemix\bundles\org.apache.servicemix.bundles.spring-context-support\3.2.11.RELEASE_1’ folder.
5. Delete data folder from ‘C:\Program Files\ArcGIS\Server\GeoEvent’.
6. Delete GeoEvent folder from ‘C:\ProgramData\Esri’.
7. Restart GeoEvent server.

As I heard, it is a known issue to ESRI INC, and they are working towards a patch to fix this.

 

Portal for ArcGIS 10.5 Fatal error “[Fatal Error] :1:1: Premature end of file”

One week after 10.5 upgrade to our test Portal for ArcGIS site, it stopped working. It was not able access the site either through Web Adaptor (80, 443) or the Portal ports (7080, 7443). The site was not running at all. Server logs showed a little evidence and was not helpful at all. I browsed through Portal for ArcGIS’s internal log files (live inside Portal for ArcGIS installation folder) and finally found something suspicious. It was the service_error.log file, located in “C:\Program Files\ArcGIS\Portal\framework\service\logs”. This log had some interesting information showed below.

Feb 10, 2017 4:27:09 PM com.esri.arcgis.portal.impl.Main start
INFO: Creating a new instance of Node Agent and starting it.
Feb 10, 2017 4:27:21 PM com.esri.arcgis.discovery.persistence.objectstore.file.FileObjectStore <init>
INFO: Initializing File Object Store with Base Directory: C:\arcgisportal\content
[Fatal Error] :1:1: Premature end of file.
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.
            at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257)
            at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)
            at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121)
            at com.esri.arcgis.portal.util.XMLUtil.openXMLDocument(XMLUtil.java:97)
            at com.esri.arcgis.portal.util.TomcatUtil.b(TomcatUtil.java:388)
            at com.esri.arcgis.portal.util.TomcatUtil.getHttpPortNumber(TomcatUtil.java:212)
            at com.esri.arcgis.portal.observers.ConfigObserver.b(ConfigObserver.java:560)
            at com.esri.arcgis.portal.observers.ConfigObserver.e(ConfigObserver.java:894)
            at com.esri.arcgis.portal.observers.ConfigObserver.beforeStart(ConfigObserver.java:184)
            at com.esri.arcgis.portal.impl.ObserverManager.beforeStart(ObserverManager.java:153)
            at com.esri.arcgis.portal.impl.NodeAgent.start(NodeAgent.java:244)
            at com.esri.arcgis.portal.impl.Main.start(Main.java:51)
Feb 10, 2017 4:27:25 PM com.esri.arcgis.portal.impl.Main start
SEVERE: com.esri.arcgis.portal.NodeAgentException: java.lang.Exception: The observer's beforeStart() function returned a failure.
            at com.esri.arcgis.portal.impl.NodeAgent.start(NodeAgent.java:279)
            at com.esri.arcgis.portal.impl.Main.start(Main.java:51)
Caused by: java.lang.Exception: The observer's beforeStart() function returned a failure.
            at com.esri.arcgis.portal.impl.NodeAgent.start(NodeAgent.java:245)
            ... 1 more

“premature end of file” normally  means a corrupted file and “Linenumber 1; columnNumber 1” means the file is empty. Further from the error message, it looks like the file should be a xml file. So an empty xml file causes the error. But which file you say.

My first thought was it is a xml file in “C:\arcgisportal\content”. Since there are thousands of iteminfo.xml and *.pkinfo files in the folder, I wrote 3 small scripts to help me looking for the corrupted xml file but ended up found nothing.
Then I thought, it only took 5 seconds for the error message to appear, it might not even run into the contents folder yet, maybe it is some configuration file in Portal for ArcGIS installation folder. A quick search found me “C:\Program Files\ArcGIS\Portal\framework\runtime\tomcat\conf\server.xml” was empty. Bingo. That’s one causing the problem. Replaced it with a standard one and the problem solved.

Further research also found me this ArcGIS for Server – Error: Failed to start the server machine. Premature end of file. Similar error and fix but for ArcGIS Server. I am wondering what let to this…

ArcGIS Data Store “data/time field value out of range” error and solution

The problem

Recently I encountered a problem while testing Portal for ArcGIS feature layer editing. I was able to add new features, update attributes and vertices but deleting features didn’t work for me. No error message after the delete button is clicked.

I have an ArcGIS Server site set as the hosting server for the Portal for ArcGIS site. ArcGIS Data Store is installed and set as the ArcGIS Server site’s managed database. Portal for ArcGIS, ArcGIS Server and ArcGIS Data Store are all 10.4.1.  The standard setting recommended by ESRI was used to set the system.

The Investigation

I tried to dig it a little deeper. To delete the feature through ArcGIS Server’s rest interface. Finally an error message appeared. Code 10500, ‘Database error has occurred’, it said.

The next step I went was checking the log files of the ArcGIS Data Store. It didn’t take long until I found the following suspicious lines.

2017-01-31 16:13:02 AEST: [5328]: [1-1] ERROR:  date/time field value out of range: "12.31.9999 23:59:59" at character 164
2017-01-31 16:13:02 AEST: [5328]: [2-1] HINT:  Perhaps you need a different "datestyle" setting.
2017-01-31 16:13:02 AEST: [5328]: [3-1] QUERY:  UPDATE hsu_9ugai.registered_sites_registered_sites SET gdb_to_date  = current_timestamp(3) AT TIME ZONE 'UTC'  WHERE objectid = old.objectid AND gdb_to_date = '12.31.9999 23:59:59'
2017-01-31 16:13:02 AEST: [5328]: [4-1] CONTEXT:  PL/pgSQL function nvv_update_78() line 15 at SQL statement
2017-01-31 16:13:02 AEST: [5328]: [5-1] STATEMENT:  DELETE FROM db_jq287.hsu_9ugai.registered_sites_registered_sites_evw WHERE objectid = $1

So an ‘date/time field value out of range’ error with a hint of solution appeared in the log file. Since ArcGIS Data Store stores data in a POSTGRESQL database. A quick GOOGLE search with the keywords would help to find the solution. The log file says ‘date/time field value out of range’ and then indicated that the out of range value is ‘12.31.9999 23:59:59’. Find the ArcGIS Data Store configuration folder, in pgdata folder, open postgresql.conf file. Search ‘datestyle’, the default setting is datestyle = ‘iso, dmy’. If you compare the format with the out of range date/time value, it is obvious they don’t match. The out of range date/time value format is ‘month.date.year’ while the ArcGIS Data Store default datestyle setting is ‘date.month.year’. That explained why the error had occurred.

The Solution

Simply change the datastyle value from ‘iso, dmy’ to ‘iso, mdy’, save the configuration file and restart ArcGIS Data Store service. Problem resolved.

ArcGIS Server 10.4.1 Sql Server SDE 10.2.2 Feature class Data source Mismatching Issue and a Temporary Solution

We encountered this issue late last year but didn’t pay too much attention to it until it happened again today.

The Backstory

We were working on a new map service on ArcGIS Server 10.4.1. After the service was successfully published, we found that the field settings on one of the layers didn’t match the relevant settings in the mxd. The layer in the mxd file had 25 fields but the relevant service layer only had 3 fields, and the total number of the records were different.

Many efforts were made to fix the issue. We tried overwriting the Map service, recreating the Map Service, etc. None of them were able to fix the issue.

What We had Found

We took the time to inspect every single file in the MapServer folder in arcgisinput folder. It didn’t take long before we found the anomaly. It was in the msd file. To inspect the content in a msd file, simply extract the msd to a folder (I used 7-Zip). There are just a bunch of xml files in the folder.

In our Sql Server SDE, we use different schemas, but occasionally we have same name feature class stored in different schema. In this particular case, COUNCILASSETS.Bridges is the feature class we want to use. However, we also have ADAC.Bridges in the same database. During the publishing process, for some reason ArcGIS Server 10.4.1 used ADAC.Bridges instead of COUNCILASSETS.Bridges as data source for the layer in the ‘bridges.xml’ shown in above image. My understanding is ArcGIS Server 10.4.1 searches the feature class by name ‘Bridges’ and uses the first result in the result list.

A Temporary Solution

When the issue is known, a solution is clear. I call it temporary because it is clearly a BUG on ArcGIS Server 10.4.1.

Fortunately, as until ArcGIS 10.5 arcpy.mapping .ConvertToMSD function is still available. Run the python function to export a msd file and replace the one that ArcGIS Server 10.4.1 is created. Remember to stop the service before deleting and replacing the original msd file.

A little More

So far we found that the output msd file is fine when using ArcGIS Desktop 10.4.1.

Also found that ArcGIS Server 10.5 has the same issue.

Esri has confirmed it is a bug affecting both 10.4.1 and 10.5 ArcGIS Server.
”BUG-000099007 : When packaging a service definition, ArcGIS 10.4.1 Desktop mixes up feature classes with the same name from two different schema”

Download an ArcGIS Server Feature Layer To A local Feature class

I think this script is quite useful. You should be able to download a feature layer (from a Dynamic Map Service or a Feature service) to a local Feature Class.

Thanks for esri arcrest module, it saves a lot of time to create the script. Without it, the script may be 10 times longer.

Just download and install arcrest and call the “SaveServerFeatureLayerToFetureclass_management” function. Parametres are explained as the following,

• out_fc: output feature class path.
• fl_url: feature layer url.
• username: if ArcGIS Server requires login, this is the username
• password: password
• token_url: if the ArcGIS Server uses a specific url for the token generator, then fill this one
• time_sleep: the time in second the script waits between two query sessions.

import arcpy, arcrest, string, random, time, tempfile, os

def MakeFGDBInUserTemp_management():
    tmp_fgdb_dir = tempfile.mkdtemp("", "arcpyex")
    tmp_fgdb_name = "tmp" + GetRandomString_general(10)
    tmp_fgdb_full_path = os.path.join(tmp_fgdb_dir, tmp_fgdb_name + ".gdb")
    arcpy.CreateFileGDB_management(tmp_fgdb_dir, tmp_fgdb_name)
    return tmp_fgdb_full_path

def GetRandomString_general(length, method=1):
    """
    This function returns a random lowercase string with the given length.
    method parameter indicates which character set to be used.
    1 selects characters from abcdefghijklmnopqrstuvwxyz.
    2 selects characters from 0123456789.
    3 selects characters from abcdefghijklmnopqrstuvwxyz and 0123456789.
    4 selects characters from abcdefghijklmnopqrstuvwxyz and 0123456789 but startswith one of the abcdefghijklmnopqrstuvwxyz.
    """
    if length  4:
        print "method is not recognizable. reset to 1."
        method = 1
    if method == 1:
        return ''.join(random.SystemRandom().choice(string.ascii_lowercase) for i in range(length))
    elif method == 2:
        return ''.join(random.SystemRandom().choice(string.digits) for i in range(length))
    elif method == 3:
        return ''.join(random.SystemRandom().choice(string.ascii_lowercase + string.digits) for i in range(length))
    elif method == 4:
        if length > 1:
            return ''.join(random.SystemRandom().choice(string.ascii_lowercase) for i in range(1)) + ''.join(random.SystemRandom().choice(string.ascii_lowercase + string.digits) for i in range(length - 1))
        else:
            return (random.SystemRandom().choice(string.ascii_lowercase) for i in range(1))

def SaveServerFeatureLayerToFetureclass_management(out_fc, fl_url, username = None, password = None, token_url = None, time_sleep=3):
    """
    This function can be used to download a ArcGIS Server feature layer to a local Feature Class.
    It copies the exposed schema and download all exposed data. The limitation is on the server side. If a layer definition query is set then the function
    can only download the data matching the definition query. If certain fields are unticked from the list, they will not be downloaded.
    """
    fgdb_path = MakeFGDBInUserTemp_management()
    arcpy.env.workspace = fgdb_path
    #get token if username and password are provided
    securityHandler = None
    if username != None and password != None:
        if token_url == None:
            token_url = fl_url.lower().split("rest")[0] + "tokens/"
        securityHandler = arcrest.AGSTokenSecurityHandler(username=username, password=password, token_url=token_url)
    #get featurelayer info
    featureLayerInfo = None
    if securityHandler != None:
        featureLayerInfo = arcrest.ags.FeatureLayer(fl_url, securityHandler=securityHandler)
    else:
        featureLayerInfo = arcrest.ags.FeatureLayer(fl_url)
    #get maxRecordCount
    maxRecordCount = featureLayerInfo.maxRecordCount
    qr = featureLayerInfo.query("1=1", "*", None, None, False, True)
    #get OBJECTID field name
    objectIdFieldName = qr["objectIdFieldName"]
    #get all available oids
    oIds = sorted(qr["objectIds"])
    #split oids into smaller lists that fits the maxRecordCount
    chunks = [oIds[x:x+maxRecordCount] for x in xrange(0, len(oIds), maxRecordCount)]
    for chunk in chunks:
        try:
            min_value = min(chunk)
            max_value = max(chunk)
            wc = "{0} >= {1} AND {0} <= {2}".format(objectIdFieldName, min_value, max_value)
            print wc 
            #query feature layer 
            q = featureLayerInfo.query(wc, "*") 
            u_name = arcpy.CreateUniqueName(GetRandomString_general(10), arcpy.env.workspace)
            #save results to a feature class 
            q.save(arcpy.env.workspace, u_name) 
            #wait some time 
            time.sleep(time_sleep) 
        except Exception as ex:
            print ex 
    fcs = arcpy.ListFeatureClasses() 
    first_fc = fcs.pop(0) 
    if len(fcs) > 0:
        arcpy.Append_management(fcs, first_fc, "NO_TEST")
    arcpy.CopyFeatures_management(os.path.join(arcpy.env.workspace, first_fc), out_fc)

SDE root takes forever to load? Check Documentation field on GBD_Items table (SDE WORKSPACE METADATA).

THE PROBLEM
Have you ever experienced this? It takes forever for ArcCatalog to load the list of items from a SDE database. In my case, it could be up to 2 minutes.

THE BACKGROUND
We had a database migration a couple of years ago from Oracle to Sql Server. A new SDE was setup for data capture and desktop viewing. We copied about 1500 feature classes, most of them sat in feature dataset, and 30 raster datasets. About 10 percent of these feature classes were versioned. Also we had compress script running on a daily basis, analyze datasets and rebuild indexes running on a weekly basis for performance concerns. After some time, we discovered the SDE was running slower and slower. It froze Catalog for about 90 to 120 seconds every time when the workspace (SDE root) was selected. However when a feature class or a raster dataset was added to ArcMap, it performed well.

THE CAUSE
It took me long time trying to figure out what the issue was. Eventually I found the xml stored in Documentation field on GDB_Items table for Workspace (Type: C673FE0F-7280-404F-8532-20755DD8FC06) was extremely large (27 Mb). When investigated a little further, I found the xml in the documentation column keeps a geoprocessing history. Every time a geoprocessing tool is ran against the SDE workspace, it updates the xml in the documentation field and keeps the detail. So it is now clear, it inserts a big record including all selected feature classes when Analyze datasets or Rebuild indexes tool runs. Over the time, the system builds a huge metadata in the Documentation field. When the SDE workspace is selected, it has to download the items list as well as the metadata, which leads to the lag.

THE SOLUTION
It is easy to fix the issue when the cause is found.
Solution 1. Remove the workspace metadata. I use sql server so the example is for sql server.

UPDATE sde.GDB_ITEMS SET Documentation = NULL WHERE [Type] = 'C673FE0F-7280-404F-8532-20755DD8FC06';

Solution 2. Remove old geoprocessing history. So this code is for Sql Server as well.

import pymssql
import xml.etree.ElementTree as ET
import time
import datetime

def ClearSDEGeoprocessingHistory_administration(host, user, password, database, treshold_days=30):
    conn = pymssql.connect(host=host, user=user, password=password, database=database)
    try:
        cur = conn.cursor()
        cur.execute("select documentation from sde.GDB_Items where type = 'C673FE0F-7280-404F-8532-20755DD8FC06'")
        new_xml_string = None
        for row in cur:
            root = ET.fromstring(row[0])
            lineage = root.findall("./Esri/DataProperties/lineage")[0]
            for process in lineage.findall("Process"):
                date_process = time.strptime(process.attrib["Date"], "%Y%m%d")
                datediff = datetime.datetime.now() - datetime.datetime.fromtimestamp(time.mktime(date_process))
                if datediff.days > treshold_days:
                    lineage.remove(process)
            new_xml_string = ET.tostring(root, encoding="utf8", method="xml")
            break
        if new_xml_string != None:
            #remove xml declaration, replace single quote with two single quotes (escape in sql server query)
            new_xml_string = ("".join(new_xml_string.split("\n")[1:])).replace("'", "''")
            cur.execute("UPDATE sde.GDB_ITEMS SET Documentation = '{0}' WHERE [Type] = 'C673FE0F-7280-404F-8532-20755DD8FC06'".format(new_xml_string))
            #you have to commit to make it actually happen
            conn.commit()
    except Exception as ex:
        print ex
    conn.close()

host = "{SQL SERVER HOST}"
user = "{DB OWNER}"
password = "{DB OWNER PWD}"
database = "{DATABSE NAME}"
#set treshold, geoprocessing history old than this will be remvoed.
treshold_days = 30

ClearSDEGeoprocessingHistory_administration(host, user, password, database)

LESSONS LEARNTED
Esri GDB keeps a track of geoprocessing history in metadata on all types of items. Consider the metadata size if you have any performance issue but are not able to locate the cause.
Esri GDB also keeps a track of copy history.

Remove Holes From Polygon FeatureClass Using ArcPy

Since the introduction of ArcPy, it became easier to manipulate Geometry using python.  During the recent work, I was involved in a project regarding fixing broken python scripts written long time ago. One of them was used to remove polygon holes base on given threshold. I rewrite the script using ArcPy and the following is the simple description and 1st edition of the code.

Original Polygon FeatureClass.

All holes are Removed. Use default threshold (0.0).

Holes smaller than Threshold are Removed.

import arcpy
def RemovePolygonHoles_management(in_fc, threshold=0.0):
    """
    The function removes holes from a polygon feature class.
    If threshold is given, only the holes smaller than the threshold will be removed.
    If no threshold is given, it removes all holes.
    in_fc is a polygon feature class.
    threshold is numeric.
    """
    desc = arcpy.Describe(in_fc)
    if desc.dataType !="FeatureClass" and desc.dataType != "ShapeFile":
        print "Invalid data type. The input is supposed to be a Polygon FeatureClass or Shapefile."
        return
    else:
        if desc.shapeType != "Polygon":
            print "The input is supposed to be a Polygon FeatureClass or Shapefile."
            return
    if threshold < 0.0:
        threshold = 0.0
    with arcpy.da.UpdateCursor(in_fc, ["SHAPE@"]) as updateCursor:
        for updateRow in updateCursor:
            shape = updateRow[0]
            new_shape = arcpy.Array()
            for part in shape:
                new_part = arcpy.Array()
                if threshold > 0:
        			#find None point in shape part
        			#in arcpy module, a None point is used to seperate exterior and interior vertices
                    null_point_index = []
                    for i in range(len(part)):
                        if part[i] == None:
                            null_point_index.append(i)
        			#if interior vertices exist, create polygons and compare polygon shape area to given threshold
        			#if larger, keep vertices, else, dismiss them
                    if len(null_point_index) > 0:
                        for k in range(0, null_point_index[0]):
                            new_part.add(part[k])
                        for i in range(len(null_point_index)):
                            pointArray = arcpy.Array()
        					#determine if the None point is the last one
                            if i+1 < len(null_point_index):
                                for j in range(null_point_index[i] + 1, null_point_index[i+1]):
                                    pointArray.add(part[j])
                            else:
                                for j in range(null_point_index[i] + 1, len(part)):
                                    pointArray.add(part[j])
        					#create a polygon to check shape area against the given threshold
                            inner_poly = arcpy.Polygon(pointArray)
        					#if larger than threshold, then add to the new part Array
                            if inner_poly.area > threshold:
                                if i+1 < len(null_point_index):
                                    for k in range(null_point_index[i], null_point_index[i+1]):
                                        new_part.add(part[k])
                                else:
                                    for k in range(null_point_index[i], len(part)):
                                        new_part.add(part[k])
                        new_shape.add(new_part)
                    #if interior does not exist, add the whole part
                    else:
                        new_shape.add(part)
                else:
                    #get the first None point index
                    first_null_point_index = 0
                    for i in range(len(part)):
                        if part[i] == None:
                            first_null_point_index = i
                            break
                    if first_null_point_index == 0:
                        new_shape.add(part)
                    else:
                        for j in range(first_null_point_index):
                            new_part.add(part[j])
                        new_shape.add(new_part)
            if len(new_shape) > 0:
                new_poly = arcpy.Polygon(new_shape)
                updateRow[0] = new_poly
                updateCursor.updateRow(updateRow)

A smart way to use ESRI “FeatureClassToFeatureClass” tool to create empty shell from given Feature class

There are many ways to create a empty shell from a given feature class using ESRI tools. But the most efficient way I know so far is using FeatureClassToFeatureClass. Simply add a expression that is never true such as ‘1>1’ or ‘1=2’. The tool creates a new feature class which inherits all but data from the given feature class.

arcpy.FeatureClassToFeatureClass(in_fc, out_location, out_name, “1>1”)