Можно ли извлечь содержание из документа COC без контура или закладки, используя Apache Tika? - PullRequest
Новая
       50

Можно ли извлечь содержание из документа COC без контура или закладки, используя Apache Tika?

1 голос
/ 10 ноября 2019

Я смотрю парсер для pdf для извлечения оглавления, чтобы определить начальный и конечный раздел

В настоящее время я использовал Apache Tika для извлечения текста из pdf 

'CONTENTS \n'
 ' \n'
 '1\xa0 INTERPRETATION AND DEFINITIONS '
 '................................................................. 5\xa0\n'
 '2\xa0 CLAUSE REFERENCES '
 '......................................................................................... '
 '11\xa0\n'
 '3\xa0 PRODUCTS AND SERVICES TO BE PROVIDED BY CONTRACTORS ....... 12\xa0\n'
 '3A\xa0 NETWORK BANDWIDTH CONNECTIVITY '
 '......................................................... 12\xa0\n'
 '4\xa0 TERMS OF PAYMENT '
 '............................................................................................ '
 '13\xa0\n'
 '5\xa0 TAXES, FEES AND DUTIES '
 '.................................................................................. '
 '14\xa0\n'
 '6\xa0 TIME FOR PERFORMANCE '
 '.................................................................................. '
 '15\xa0\n'
 "8\xa0 CONTRACTOR'S OBLIGATIONS "
 '.......................................................................... '
 '17\xa0\n'
 '9\xa0 RESPONSIBILITY FOR THE '
 'SYSTEM................................................................. '
 '18\xa0\n'
 '10\xa0 MODIFICATION OF SYSTEM '
 '................................................................................ '
 '19\xa0\n'
 '11\xa0 SECURITY DEPOSIT '
 '............................................................................................... '
 '19\xa0\n'
 '12\xa0 PROJECT MANAGEMENT '
 '..................................................................................... '
 '20\xa0\n'
 '\n'
 '12.1\xa0 Intentionally Left Blank '
 '............................................................................................................ '
 '20\xa0\n'
 '12.2\xa0 Project '
 'Office............................................................................................................................. '
 '20\xa0\n'
 '12.3\xa0 Project Manager and Other Personnel '
 '................................................................................. '
 '20\xa0\n'
 '12.4\xa0 Project Plan '
 '............................................................................................................................... '
 '21\xa0\n'
 '12.5\xa0 Monthly Progress Reports & Meeting '
 '................................................................................... '
 '21\xa0\n'
 '\n'
 "13\xa0 CONTRACTOR'S PERSONNEL "
 '............................................................................ '
 '22\xa0\n'
 '14\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '23\xa0\n'
 '15\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '23\xa0\n'
 '16\xa0 INFORMATION AND ACCESS '
 '.............................................................................. '
 '23\xa0\n'
 '17\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '23\xa0\n'
 '18\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '23\xa0\n'
 '19\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '23\xa0\n'
 '20\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '23\xa0\n'
 '21\xa0 INSTALLATION '
 '......................................................................................................... '
 '23\xa0\n'
 '22\xa0 ACCEPTANCE TESTS '
 '............................................................................................ '
 '23\xa0\n'
 '\n'
 '22.1\xa0 Conducting Acceptance Tests '
 '............................................................................................... '
 '23\xa0\n'
 '22.2\xa0 Notice of Commencement and Completion of Acceptance Tests '
 '.................................. 24\xa0\n'
 '22.3\xa0 Delay in Acceptance Tests '
 '..................................................................................................... '
 '25\xa0\n'
 '22.4\xa0 Intentionally Left Blank '
 '............................................................................................................ '
 '25\xa0\n'
 '22.5\xa0 Software Installation Tests '
 '..................................................................................................... '
 '25\xa0\n'
 '22.6\xa0 System Performance Tests '
 '.................................................................................................... '
 '26\xa0\n'
 '22.7\xa0 Failure of Acceptance Tests '
 '................................................................................................... '
 '27\xa0\n'
 '22.8\xa0 Commissioning Date '
 '................................................................................................................ '
 '27\xa0\n'
 '\n'
 '23\xa0 LIQUIDATED DAMAGES FOR LATE RELEASE/COMMISSIONING .............. '
 '28\xa0\n'
 '24\xa0 PERFORMANCE GUARANTEE PERIOD '
 '............................................................ 29\xa0\n'
 '25.\xa0 SYSTEM WARRANTY PERIOD '
 '............................................................................. '
 '31\xa0\n'
 '26\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '32\xa0\n'
 '27\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '32\xa0\n'
 '28\xa0 TRAINING '
 '.................................................................................................................. '
 '32\xa0\n'
 '30\xa0 OWNERSHIP OF INTELLECTUAL PROPERTY '
 '................................................. 33\xa0\n'
 '31\xa0 UNAUTHORISED CODE '
 '......................................................................................... '
 '34\xa0\n'
 '32\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '35\xa0\n'
 '33\xa0 DOCUMENTATION '
 '.................................................................................................. '
 '36\xa0\n'
 '34\xa0 LIABILITY OF CONTRACTOR '
 '............................................................................... '
 '36\xa0\n'
 '35\xa0 INTELLECTUAL PROPERTY INDEMNITY '
 '.......................................................... 36\xa0\n'
 '36\xa0 RELOCATION OF SYSTEM '
 '................................................................................... '
 '37\xa0\n'
 '\n'
 '\n'
 '\n'
 ' CONDITIONS OF CONTRACT   \n'
 ' \n'
 '\n'
 ' \n'
 ' 3 Version 2.4 \n'
 '\n'
 '37\xa0 LANGUAGE '
 '............................................................................................................... '
 '38\xa0\n'
 '38\xa0 LOSSES '
 '...................................................................................................................... '
 '38\xa0\n'
 '39\xa0 LIMITATION OF LIABILITY '
 '..................................................................................... '
 '39\xa0\n'
 '40.\xa0 INSURANCE FOR PERSONAL INJURY, WORK INJURY COMPENSATION \n'
 'AND PROPERTY DAMAGE '
 '................................................................................................ '
 '40\xa0\n'
 '41\xa0 CONFIDENTIALITY '
 '.................................................................................................. '
 '43\xa0\n'
 '41A\xa0 DATA SECURITY AND PROTECTION '
 '................................................................. 44\xa0\n'
 '42\xa0 COMPLIANCE WITH LAW '
 '...................................................................................... '
 '44\xa0\n'
 '43\xa0 SUB-CONTRACT, ASSIGNMENT, TRANSFER '
 '................................................. 45\xa0\n'
 '44\xa0 FORCE MAJEURE '
 '................................................................................................... '
 '45\xa0\n'
 '45\xa0 PUBLIC RELEASE OF INFORMATION '
 '................................................................ 46\xa0\n'
 '46\xa0 GIFTS, INDUCEMENT AND REWARDS '
 '.............................................................. 46\xa0\n'
 '47\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '47\xa0\n'
 '48\xa0 VARIATION AND VALUATION OF VARIATIONS '
 '............................................... 47\xa0\n'
 '49\xa0 WAIVER '
 '...................................................................................................................... '
 '51\xa0\n'
 '50\xa0 TERMINATION OF CONTRACT '
 '............................................................................ '
 '51\xa0\n'
 '51\xa0 POLICY, SECURITY AND AUDIT '
 '.......................................................................... '
 '54\xa0\n'
 '\n'
 '51.1\xa0 Policy '
 '.......................................................................................................................................... '
 '54\xa0\n'
 '51.1A\xa0 Security '
 '................................................................................................................................. '
 '55\xa0\n'
 '51.2\xa0 Audit '
 '............................................................................................................................................ '
 '55\xa0\n'
 '\n'
 '51A\xa0 SECURITY AND DATA BREACH PROCEDURES '
 '............................................. 55\xa0\n'
 '51B.\xa0 SECURITY VULNERABILITIES '
 '............................................................................. '
 '57\xa0\n'
 '52\xa0 GOVERNING LAW '
 '................................................................................................... '
 '59\xa0\n'
 '53\xa0 INTENTIONALY LEFT BLANK '
 '............................................................................... '
 '59\xa0\n'
 '54\xa0 SETTLEMENT OF DISPUTE '
 '.................................................................................. '
 '59\xa0\n'
 '55\xa0 CORRESPONDENCE '
 '.............................................................................................. '
 '61\xa0\n'
 '56\xa0 REMEDIES '
 '................................................................................................................. '
 '61\xa0\n'
 '56A\xa0 HANDLING OF HDB’S HARDWARE AND SOFTWARE '
 '................................... 62\xa0\n'
 '57\xa0 CLAIMS FOR EXTRA WORK '
 '................................................................................. '
 '62\xa0\n'
 '58\xa0 RIGHTS OF THIRD PARTIES '
 '................................................................................ '
 '62\xa0\n'
 '59\xa0 GOVERNMENT ELECTRONIC BUSINESS (GEBIZ) '
 '......................................... 63\xa0\n'
 '60\xa0 CONSORTIUM '
 '.......................................................................................................... '
 '63\xa0\n'
 '61\xa0 OWNERSHIP OF DOCUMENTATION AND DISPOSAL OF \n'
 'DOCUMENTATION UPON TERMINATION OF CONTRACT OR COMPLETION OF \n'
 'CONTRACT '
 '........................................................................................................................... '
 '64\xa0\n'
 '62\xa0 COEXISTENCE STRATEGY '
 '.................................................................................. '
 '64\xa0\n'
 '63\xa0 SET-OFF '
 '.................................................................................................................... '
 '65\xa0\n'
 '64\xa0 ENTIRE AND WHOLE AGREEMENT '
 '................................................................... 65\xa0\n'
 '65\xa0 SURVIVAL '
 '.................................................................................................................. '
 '65\xa0\n'
 '66\xa0 ISSUANCE OF WRITTEN WARNING BY THE HDB '
 '.......................................... 65\xa0\n'
 '67\xa0 LIQUIDATED DAMAGES FOR DATA AND SECURITY BREACHES ............. '
 '66\xa0\n'
 '68\xa0 INTENTIONALLY LEFT BLANK '
 '............................................................................. '
 '66\xa0\n'
 '69\xa0 SEVERABILITY '
 '......................................................................................................... '
 '66\xa0\n'
 '70\xa0 ORDER OF '
 'PRECEDENCE.................................................................................... '
 '67\xa0\n'
 '70A.\xa0 SCHEDULES/ANNEXES/APPENDICES '
 '.............................................................. 67\xa0\n'
 '71\xa0 NUISANCE AND IRREGULARITIES '
 '..................................................................... '
 '67\xa0\n'
 '72\xa0 NOVATION OF CONTRACT '
 '................................................................................... '
 '68\xa0\n'
 ' \n'

Я попробовал функцию gettoc, однако, вывод возвращает 0, так как я подозреваю, что в pdf нет схемы содержания. 

file = '/Users/admin/PycharmProjects/text_extraction/05 Conditions of Contract_HDB.pdf'
from pdf_layout_scanner import layout_scanner
toc=layout_scanner.get_toc(file)
print(len(toc))

Я ожидал, что как только мы обнаружим таблицу содержимого в pdf, она вернет первый разделкак: 

 '1\xa0 INTERPRETATION AND DEFINITIONS '

и следующий как последний раздел: 

 '72\xa0 NOVATION OF CONTRACT '
...